Mercurial Hosting > traffic-intelligence

annotate trafficintelligence/storage.py @ 1257:e59a0a475a0a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
removed bug with unnecessary data
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Wed, 03 Apr 2024 16:39:41 -0400
parents 56d0195d043e
children 3d6ee243d5c0 158eee1aeb21
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1 #! /usr/bin/env python
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
2 # -*- coding: utf-8 -*-
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
4
1003
75af46516b2b work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 998
diff changeset
5 from pathlib import Path
1028
cc5cb04b04b0 major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1026
diff changeset
6 import shutil
919
7b3f2e0a2652 saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 918
diff changeset
7 from copy import copy
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
8 import sqlite3, logging
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
9
1215
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
10 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt, floor as npfloor, ceil as npceil, linalg, int32, int64, reshape, dot, vstack, transpose, ones, zeros_like, pi, NaN
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
11 from pandas import read_csv, merge, concat, DataFrame
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
12
1233
d5695e0b59d9 saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1232
diff changeset
13 from trafficintelligence import utils, moving, events, indicators
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
14 from trafficintelligence.base import VideoFilenameAddable
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
15
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
16
50
7b06d649122b re-arrangement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 48
diff changeset
17 ngsimUserTypes = {'twowheels':1,
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
18 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
19 'truck':3}
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
20
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
21 tableNames = {'feature':'positions',
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
22 'object': 'objects',
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
23 'objectfeatures': 'positions'}
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
24
1049
c9c03c97ed9f bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1044
diff changeset
25 sqlite3.register_adapter(int64, lambda val: int(val))
c9c03c97ed9f bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1044
diff changeset
26 sqlite3.register_adapter(int32, lambda val: int(val))
c9c03c97ed9f bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1044
diff changeset
27
259
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
28 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
29 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
30 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
31
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
32 # utils
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
33 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
34 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
35
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
36 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
37 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
38 try:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
39 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
40 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
41 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
42 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
43 printDBError(error)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
44
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
45 def deleteFromSqlite(filename, dataType):
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
46 'Deletes (drops) some tables in the filename depending on type of data'
1003
75af46516b2b work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 998
diff changeset
47 if Path(filename).is_file():
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
48 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
49 if dataType == 'object':
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
50 dropTables(connection, ['objects', 'objects_features'])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
51 elif dataType == 'interaction':
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
52 dropTables(connection, ['interactions', 'indicators'])
1254
a477ad82ab66 minor addition to drop tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1253
diff changeset
53 elif dataType == 'curvilinear':
a477ad82ab66 minor addition to drop tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1253
diff changeset
54 dropTables(connection, ['curvilinear_positions'])
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
55 elif dataType == 'bb':
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
56 dropTables(connection, ['bounding_boxes'])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
57 elif dataType == 'pois':
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
58 dropTables(connection, ['gaussians2d', 'objects_pois'])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
59 elif dataType == 'prototype':
1035
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
60 dropTables(connection, ['prototypes', 'objects_prototypes', 'features_prototypes'])
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
61 else:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
62 print('Unknown data type {} to delete from database'.format(dataType))
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
63 else:
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
64 print('{} does not exist'.format(filename))
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
65
919
7b3f2e0a2652 saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 918
diff changeset
66 def tableExists(connection, tableName):
740
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
67 'indicates if the table exists in the database'
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
68 try:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
69 cursor = connection.cursor()
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
70 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'')
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
71 return cursor.fetchone()[0] == 1
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
72 except sqlite3.OperationalError as error:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
73 printDBError(error)
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
74
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
75 def tableNames(filename):
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
76 'Lists the names of the tables in the SQLite file'
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
77 if Path(filename).is_file():
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
78 with sqlite3.connect(filename) as connection:
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
79 try:
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
80 cursor = connection.cursor()
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
81 cursor.execute('SELECT name FROM sqlite_master WHERE type = \'table\'')
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
82 return [row[0] for row in cursor]
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
83 except sqlite3.OperationalError as error:
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
84 printDBError(error)
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
85 return []
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
86
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
87 def createTrajectoryTable(cursor, tableName):
919
7b3f2e0a2652 saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 918
diff changeset
88 if tableName.endswith('positions') or tableName.endswith('velocities'):
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
89 cursor.execute("CREATE TABLE IF NOT EXISTS "+tableName+" (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
90 else:
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
91 print('Unallowed name {} for trajectory table'.format(tableName))
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
92
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
93 def createObjectsTable(cursor):
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
94 cursor.execute("CREATE TABLE IF NOT EXISTS objects (object_id INTEGER, road_user_type INTEGER, n_objects INTEGER, PRIMARY KEY(object_id))")
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
95
979
cc89267b5ff9 work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 978
diff changeset
96 def createAssignmentTable(cursor, objectType1, objectType2, objectIdColumnName1, objectIdColumnName2):
cc89267b5ff9 work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 978
diff changeset
97 cursor.execute("CREATE TABLE IF NOT EXISTS "+objectType1+"s_"+objectType2+"s ("+objectIdColumnName1+" INTEGER, "+objectIdColumnName2+" INTEGER, PRIMARY KEY("+objectIdColumnName1+","+objectIdColumnName2+"))")
913
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
98
979
cc89267b5ff9 work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 978
diff changeset
99 def createObjectsFeaturesTable(cursor):
913
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
100 cursor.execute("CREATE TABLE IF NOT EXISTS objects_features (object_id INTEGER, trajectory_id INTEGER, PRIMARY KEY(object_id, trajectory_id))")
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
101
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
102
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
103 def createCurvilinearTrajectoryTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
104 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
105
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
106 def createFeatureCorrespondenceTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
107 cursor.execute('CREATE TABLE IF NOT EXISTS feature_correspondences (trajectory_id INTEGER, source_dbname VARCHAR, db_trajectory_id INTEGER, PRIMARY KEY(trajectory_id))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
108
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
109 def createInteractionTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
110 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
111
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
112 def createIndicatorTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
113 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
114
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
115 def insertTrajectoryQuery(tableName):
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
116 return "INSERT INTO "+tableName+" VALUES (?,?,?,?)"
831
a8ff35e6fb43 forgotten change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 830
diff changeset
117
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
118 def insertObjectQuery():
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
119 return "INSERT INTO objects VALUES (?,?,?)"
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
120
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
121 def insertObjectFeatureQuery():
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
122 return "INSERT INTO objects_features VALUES (?,?)"
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
123
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
124 def createIndex(connection, tableName, columnName, unique = False):
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
125 '''Creates an index for the column in the table
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
126 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
127 try:
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
128 cursor = connection.cursor()
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
129 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
130 if unique:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
131 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
132 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
133 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
134 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
135 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
136 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
137
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
138 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
139 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
140 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
141 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
142 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
143 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
144 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
145 cursor.execute("SELECT COUNT(*) from "+tableName)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
146 else:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
147 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
148 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
149 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
150 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
151
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
152 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
153 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
154 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
155 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
156 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
157 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
158 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
159 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
160 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
161 else:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
162 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
163 return cursor.fetchone()[0]
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
164 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
165 printDBError(error)
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
166
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
167 def getObjectCriteria(objectNumbers):
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
168 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
169 query = ''
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
170 elif type(objectNumbers) == int:
852
45a53542e046 updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 850
diff changeset
171 query = '<= {0}'.format(objectNumbers-1)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
172 elif type(objectNumbers) == list:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
173 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
174 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
175 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
176 query = ''
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
177 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
178
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
179 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None, timeStep = None):
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
180 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
181 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
182
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
183 returns a moving object'''
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
184 cursor = connection.cursor()
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
185
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
186 try:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
187 objectCriteria = getObjectCriteria(objectNumbers)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
188 queryStatement = None
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
189 if trajectoryType == 'feature':
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
190 queryStatement = 'SELECT * from '+tableName
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
191 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
192 queryStatement += ' WHERE trajectory_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
193 elif objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
194 queryStatement += ' WHERE trajectory_id '+objectCriteria
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
195 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
196 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
197 queryStatement += ' ORDER BY trajectory_id, frame_number'
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
198 elif trajectoryType == 'object':
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
199 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
200 if objectNumbers is not None:
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
201 queryStatement += ' AND OF.object_id '+objectCriteria
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
202 if timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
203 queryStatement += ' AND P.frame_number%{} = 0'.format(timeStep)
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
204 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
205 elif trajectoryType in ['bbtop', 'bbbottom']:
587
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
206 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
207 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
208 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
209 corner = 'bottom_right'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
210 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
211 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
212 queryStatement += ' WHERE object_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
213 elif objectNumbers is not None:
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
214 queryStatement += ' WHERE object_id '+objectCriteria
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
215 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
216 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
217 queryStatement += ' ORDER BY object_id, frame_number'
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
218 else:
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
219 print('Unknown trajectory type {}'.format(trajectoryType))
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
220 if queryStatement is not None:
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
221 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
222 logging.debug(queryStatement)
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
223 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
224 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
225 return []
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
226
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
227 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
228 obj = None
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
229 objects = []
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
230 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
231 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
232 objId = row[0]
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
233 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
234 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
235 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
236 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
237 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
238 else:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
239 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
240 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
241
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
242 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
243 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
244 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
245 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
246
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
247 return objects
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
248
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
249 def loadObjectAttributesFromTable(cursor, objectNumbers, loadNObjects = False):
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
250 objectCriteria = getObjectCriteria(objectNumbers)
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
251 queryStatement = 'SELECT object_id, road_user_type'
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
252 if loadNObjects:
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
253 queryStatement += ', n_objects'
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
254 queryStatement += ' FROM objects'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
255 if objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
256 queryStatement += ' WHERE object_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
257 cursor.execute(queryStatement)
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
258 attributes = {}
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
259 if loadNObjects:
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
260 for row in cursor:
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
261 attributes[row[0]] = row[1:]
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
262 else:
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
263 for row in cursor:
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
264 attributes[row[0]] = row[1]
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
265 return attributes
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
266
1044
75a6ad604cc5 work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1041
diff changeset
267 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False, timeStep = None, nLongestFeaturesPerObject = None):
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
268 '''Loads the trajectories (in the general sense,
1044
75a6ad604cc5 work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1041
diff changeset
269 either features, objects (feature groups), longest features per object, or bounding box series)
75a6ad604cc5 work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1041
diff changeset
270 types are only feature or object
75a6ad604cc5 work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1041
diff changeset
271 if object, features can be loaded with withFeatures or nLongestObjectFeatures used to select the n longest features
1040
20799ac9524e integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1039
diff changeset
272
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
273 The number loaded is either the first objectNumbers objects,
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
274 or the indices in objectNumbers from the database'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
275 objects = []
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
276 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
277 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
278 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers, timeStep)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
279 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers, timeStep)
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
280
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
281 if len(objectVelocities) > 0:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
282 for o,v in zip(objects, objectVelocities):
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
283 if o.getNum() == v.getNum():
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
284 o.velocities = v.positions
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
285 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
286 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
287 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
288
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
289 if trajectoryType == 'object':
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
290 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
291 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
292 # attribute feature numbers to objects
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
293 queryStatement = 'SELECT trajectory_id, object_id FROM objects_features'
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
294 if objectNumbers is not None:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
295 queryStatement += ' WHERE object_id '+getObjectCriteria(objectNumbers)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
296 queryStatement += ' ORDER BY object_id' # order is important to group all features per object
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
297 logging.debug(queryStatement)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
298 cursor.execute(queryStatement)
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
299
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
300 featureNumbers = {}
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
301 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
302 objId = row[1]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
303 if objId not in featureNumbers:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
304 featureNumbers[objId] = [row[0]]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
305 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
306 featureNumbers[objId].append(row[0])
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
307
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
308 for obj in objects:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
309 obj.featureNumbers = featureNumbers[obj.getNum()]
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
310
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
311 # load userType
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
312 attributes = loadObjectAttributesFromTable(cursor, objectNumbers, True)
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
313 for obj in objects:
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
314 userType, nObjects = attributes[obj.getNum()]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
315 obj.setUserType(userType)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
316 obj.setNObjects(nObjects)
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
317
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
318 # add features
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
319 if withFeatures:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
320 for obj in objects:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
321 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', obj.featureNumbers, timeStep = timeStep)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
322 elif nLongestFeaturesPerObject is not None:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
323 for obj in objects:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
324 queryStatement = 'SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions WHERE trajectory_id '+getObjectCriteria(obj.featureNumbers)+' GROUP BY trajectory_id ORDER BY length DESC'
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
325 logging.debug(queryStatement)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
326 cursor.execute(queryStatement)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
327 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', [row[0] for i,row in enumerate(cursor) if i<nLongestFeaturesPerObject], timeStep = timeStep)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
328
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
329 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
330 printDBError(error)
1178
ee3eaf902b83 minor improvement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1177
diff changeset
331 else:
ee3eaf902b83 minor improvement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1177
diff changeset
332 print('Impossible to load from non-existing file '+filename)
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
333 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
334
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
335 def loadObjectFeatureFrameNumbers(filename, objectNumbers = None):
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
336 'Loads the feature frame numbers for each object'
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
337 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
338 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
339 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
340 queryStatement = 'SELECT OF.object_id, TL.trajectory_id, TL.length FROM (SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions GROUP BY trajectory_id) TL, objects_features OF WHERE TL.trajectory_id = OF.trajectory_id'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
341 if objectNumbers is not None:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
342 queryStatement += ' AND object_id '+getObjectCriteria(objectNumbers)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
343 queryStatement += ' ORDER BY OF.object_id, TL.length DESC'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
344 logging.debug(queryStatement)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
345 cursor.execute(queryStatement)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
346 objectFeatureNumbers = {}
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
347 for row in cursor:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
348 objId = row[0]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
349 if objId in objectFeatureNumbers:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
350 objectFeatureNumbers[objId].append(row[1])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
351 else:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
352 objectFeatureNumbers[objId] = [row[1]]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
353 return objectFeatureNumbers
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
354 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
355 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
356 return None
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
357
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
358 def addCurvilinearTrajectoriesFromSqlite(filename, objects):
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
359 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane)
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
360 from a database to an existing MovingObject dict (indexed by each objects's num)'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
361 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
362 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
363
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
364 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
365 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number')
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
366 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
367 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
368 return []
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
369
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
370 missingObjectNumbers = []
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
371 objNum = None
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
372 for row in cursor:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
373 if objNum != row[0]:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
374 objNum = row[0]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
375 if objNum in objects:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
376 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
377 else:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
378 missingObjectNumbers.append(objNum)
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
379 if objNum in objects:
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
380 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
381 if len(missingObjectNumbers) > 0:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
382 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers))
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
383
1039
5621e4ad2428 removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1037
diff changeset
384 def saveTrajectoriesToTable(connection, objects, trajectoryType):
918
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
385 'Saves trajectories in table tableName'
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
386 cursor = connection.cursor()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
387 # Parse feature and/or object structure and commit to DB
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
388 if(trajectoryType == 'feature' or trajectoryType == 'object'):
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
389 # Extract features from objects
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
390 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
391 features = []
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
392 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
393 if obj.hasFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
394 features += obj.getFeatures()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
395 if len(features) == 0:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
396 print('Warning, objects have no features') # todo save centroid trajectories?
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
397 elif trajectoryType == 'feature':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
398 features = objects
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
399 # Setup feature queries
1039
5621e4ad2428 removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1037
diff changeset
400 createTrajectoryTable(cursor, "positions")
5621e4ad2428 removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1037
diff changeset
401 createTrajectoryTable(cursor, "velocities")
5621e4ad2428 removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1037
diff changeset
402 positionQuery = insertTrajectoryQuery("positions")
5621e4ad2428 removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1037
diff changeset
403 velocityQuery = insertTrajectoryQuery("velocities")
918
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
404 # Setup object queries
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
405 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
406 createObjectsTable(cursor)
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
407 createObjectsFeaturesTable(cursor)
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
408 objectQuery = insertObjectQuery()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
409 objectFeatureQuery = insertObjectFeatureQuery()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
410 for feature in features:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
411 num = feature.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
412 frameNum = feature.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
413 for p in feature.getPositions():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
414 cursor.execute(positionQuery, (num, frameNum, p.x, p.y))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
415 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
416 velocities = feature.getVelocities()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
417 if velocities is not None:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
418 frameNum = feature.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
419 for v in velocities[:-1]:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
420 cursor.execute(velocityQuery, (num, frameNum, v.x, v.y))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
421 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
422 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
423 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
424 if obj.hasFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
425 for feature in obj.getFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
426 featureNum = feature.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
427 cursor.execute(objectFeatureQuery, (obj.getNum(), featureNum))
1052
1748c02f9ac3 Modifying storage.py [configloaders]
Wendlasida
parents: 1041
diff changeset
428 cursor.execute(objectQuery, (obj.getNum(), obj.getUserType(), obj.nObjects if hasattr(obj, 'nObjects') and obj.nObjects is not None else 1))
918
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
429 # Parse curvilinear position structure
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
430 elif(trajectoryType == 'curvilinear'):
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
431 createCurvilinearTrajectoryTable(cursor)
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
432 curvilinearQuery = "INSERT INTO curvilinear_positions VALUES (?,?,?,?,?)"
918
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
433 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
434 num = obj.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
435 frameNum = obj.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
436 for p in obj.getCurvilinearPositions():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
437 cursor.execute(curvilinearQuery, (num, frameNum, p[0], p[1], p[2]))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
438 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
439 else:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
440 print('Unknown trajectory type {}'.format(trajectoryType))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
441 connection.commit()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
442
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 917
diff changeset
443 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType):
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
444 '''Writes features, ie the trajectory positions (and velocities if exist)
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
445 with their instants to a specified sqlite file
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
446 Either feature positions (and velocities if they exist)
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
447 or curvilinear positions will be saved at a time'''
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
448
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
449 with sqlite3.connect(outputFilename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
450 try:
1040
20799ac9524e integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1039
diff changeset
451 saveTrajectoriesToTable(connection, objects, trajectoryType)
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
452 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
453 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
454
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
455 def setRoadUserTypes(filename, objects):
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
456 '''Saves the user types of the objects in the sqlite database stored in filename
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
457 The objects should exist in the objects table'''
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
458 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
459 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
460 for obj in objects:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
461 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum()))
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
462 connection.commit()
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
463
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
464 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None, timeStep = None):
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
465 '''Loads bounding box moving object from an SQLite
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
466 (format of SQLite output by the ground truth annotation tool
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
467 or Urban Tracker
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
468
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
469 Load descriptions?'''
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
470 objects = []
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
471 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
472 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
473 if objectType == 'bb':
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
474 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers, timeStep)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
475 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers, timeStep)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
476 userTypes = loadObjectAttributesFromTable(connection.cursor(), objectNumbers) # string format is same as object
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
477
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
478 for t, b in zip(topCorners, bottomCorners):
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
479 num = t.getNum()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
480 if t.getNum() == b.getNum():
1133
c4d9c270f999 modification for performance computation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1131
diff changeset
481 annotation = moving.BBMovingObject(num, t, b, t.getTimeInterval(), userTypes[num])
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
482 objects.append(annotation)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
483 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
484 print ('Unknown type of bounding box {}'.format(objectType))
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
485 return objects
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
486
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
487 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
488 roadUserNumbers = list(interaction.getRoadUserNumbers())
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
489 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
490
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
491 def saveInteractionsToSqlite(filename, interactions):
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
492 'Saves the interactions in the table'
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
493 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
494 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
495 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
496 createInteractionTable(cursor)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
497 for inter in interactions:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
498 saveInteraction(cursor, inter)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
499 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
500 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
501 connection.commit()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
502
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
503 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
504 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
505 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
506 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
507
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
508 def saveIndicatorsToSqlite(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
509 'Saves the indicator values in the table'
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
510 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
511 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
512 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
513 createInteractionTable(cursor)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
514 createIndicatorTable(cursor)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
515 for inter in interactions:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
516 saveInteraction(cursor, inter)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
517 for indicatorName in indicatorNames:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
518 indicator = inter.getIndicator(indicatorName)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
519 if indicator is not None:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
520 saveIndicator(cursor, inter.getNum(), indicator)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
521 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
522 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
523 connection.commit()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
524
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
525 def loadInteractionsFromSqlite(filename):
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
526 '''Loads interaction and their indicators
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
527
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
528 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
529 interactions = []
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
530 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
531 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
532 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
533 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
534 cursor.execute('SELECT INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
535 interactionNum = -1
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
536 indicatorTypeNum = -1
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
537 tmpIndicators = {}
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
538 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
539 if row[0] != interactionNum:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
540 interactionNum = row[0]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
541 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
542 interactions[-1].indicators = {}
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
543 if indicatorTypeNum != row[5] or row[0] != interactionNum:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
544 indicatorTypeNum = row[5]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
545 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
546 indicatorValues = {row[6]:row[7]}
1182
0e5d37b0b9ff bug corrections
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1178
diff changeset
547 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.mostSevereIsMinIndicators)
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
548 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
549 indicatorValues[row[6]] = row[7]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
550 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
551 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
552 printDBError(error)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
553 return []
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
554 return interactions
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
555 # load first and last object instants
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
556 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
557
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
558 def createBoundingBoxTable(filename, invHomography = None):
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
559 '''Create the table to store the object bounding boxes in image space
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
560 '''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
561 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
562 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
563 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
564 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
565 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
566 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from '
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
567 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
568 'GROUP BY object_id, frame_number')
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
569 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
570 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
571 connection.commit()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
572
586
ff4f0ce46ca6 modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 585
diff changeset
573 def loadBoundingBoxTableForDisplay(filename):
647
458890c0437c cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 646
diff changeset
574 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
575 boundingBoxes = {} # list of bounding boxes for each instant
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
576 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
577 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
578 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
579 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
580 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
581 result = cursor.fetchall()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
582 if len(result) > 0:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
583 cursor.execute('SELECT * FROM bounding_boxes')
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
584 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
585 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
586 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
587 printDBError(error)
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
588 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
589
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
590 #########################
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
591 # saving and loading for scene interpretation: POIs and Prototypes
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
592 #########################
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
593
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
594 def savePrototypesToSqlite(filename, prototypes):
1033
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
595 '''save the prototypes (a prototype is defined by a filename, a number (id) and type)'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
596 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
597 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
598 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
599 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), nmatchings INTEGER, PRIMARY KEY (prototype_filename, prototype_id, trajectory_type))')
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
600 for p in prototypes:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
601 cursor.execute('INSERT INTO prototypes VALUES(?,?,?,?)', (p.getFilename(), p.getNum(), p.getTrajectoryType(), p.getNMatchings()))
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
602 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
603 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
604 connection.commit()
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
605
1033
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
606 def setPrototypeMatchingsInSqlite(filename, prototypes):
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
607 '''updates the prototype matchings'''
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
608 with sqlite3.connect(filename) as connection:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
609 cursor = connection.cursor()
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
610 try:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
611 for p in prototypes:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
612 if p.getNMatchings() is None:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
613 nMatchings = 'NULL'
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
614 else:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
615 nMatchings = p.getNMatchings()
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
616 cursor.execute('UPDATE prototypes SET nmatchings = {} WHERE prototype_filename = \"{}\" AND prototype_id = {} AND trajectory_type = \"{}\"'.format(nMatchings, p.getFilename(), p.getNum(), p.getTrajectoryType()))
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
617 except sqlite3.OperationalError as error:
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
618 printDBError(error)
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
619 connection.commit()
8ffb3ae9f3d2 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1030
diff changeset
620
1037
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
621 def prototypeAssignmentNames(objectType):
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
622 tableName = objectType+'s_prototypes'
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
623 if objectType == 'feature':
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
624 #tableName = 'features_prototypes'
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
625 objectIdColumnName = 'trajectory_id'
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
626 elif objectType == 'object':
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
627 #tableName = 'objects_prototypes'
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
628 objectIdColumnName = 'object_id'
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
629 return tableName, objectIdColumnName
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
630
1035
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
631 def savePrototypeAssignmentsToSqlite(filename, objectNumbers, objectType, labels, prototypes):
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
632 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
633 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
634 try:
1037
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
635 tableName, objectIdColumnName = prototypeAssignmentNames(objectType)
979
cc89267b5ff9 work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 978
diff changeset
636 cursor.execute('CREATE TABLE IF NOT EXISTS '+tableName+' ('+objectIdColumnName+' INTEGER, prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), PRIMARY KEY('+objectIdColumnName+', prototype_filename, prototype_id, trajectory_type))')
1035
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
637 for objNum, label in zip(objectNumbers, labels):
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
638 if label >=0:
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
639 proto = prototypes[label]
933588568bec major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1033
diff changeset
640 cursor.execute('INSERT INTO '+tableName+' VALUES(?,?,?,?)', (objNum, proto.getFilename(), proto.getNum(), proto.getTrajectoryType()))
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
641 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
642 printDBError(error)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
643 connection.commit()
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
644
1037
6a6c37eb3a74 added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1035
diff changeset
645 def loadPrototypeAssignmentsFromSqlite(filename, objectType):
1239
31173c4699d2 minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1233
diff changeset
646 '''Loads the assignments between prototypes
31173c4699d2 minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1233
diff changeset
647 Returns a dictionary with prototypes as keys and a list of
31173c4699d2 minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1233
diff changeset
648 the objects/feature nums assigned to each'''
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
649 prototypeAssignments = {}
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
650 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
651 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
652 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
653 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
654 tableName, objectIdColumnName = prototypeAssignmentNames(objectType)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
655 cursor.execute('SELECT * FROM '+tableName)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
656 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
657 p = moving.Prototype(row[1], row[2], row[3])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
658 if p in prototypeAssignments:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
659 prototypeAssignments[p].append(row[0])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
660 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
661 prototypeAssignments[p] = [row[0]]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
662 return prototypeAssignments
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
663 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
664 printDBError(error)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
665 return prototypeAssignments
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
666
920
499154254f37 improved prototype loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 919
diff changeset
667 def loadPrototypesFromSqlite(filename, withTrajectories = True):
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
668 'Loads prototype ids and matchings (if stored)'
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
669 prototypes = []
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
670 if Path(filename).is_file():
1143
8ac52ebff5f7 avoid using chdir
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1141
diff changeset
671 parentPath = Path(filename).resolve().parent
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
672 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
673 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
674 objects = []
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
675 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
676 cursor.execute('SELECT * FROM prototypes')
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
677 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
678 prototypes.append(moving.Prototype(row[0], row[1], row[2], row[3]))
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
679 if withTrajectories:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
680 for p in prototypes:
1143
8ac52ebff5f7 avoid using chdir
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1141
diff changeset
681 p.setMovingObject(loadTrajectoriesFromSqlite(str(parentPath/p.getFilename()), p.getTrajectoryType(), [p.getNum()])[0])
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
682 # loadingInformation = {} # complicated slightly optimized
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
683 # for p in prototypes:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
684 # dbfn = p.getFilename()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
685 # trajType = p.getTrajectoryType()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
686 # if (dbfn, trajType) in loadingInformation:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
687 # loadingInformation[(dbfn, trajType)].append(p)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
688 # else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
689 # loadingInformation[(dbfn, trajType)] = [p]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
690 # for k, v in loadingInformation.iteritems():
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
691 # objects += loadTrajectoriesFromSqlite(k[0], k[1], [p.getNum() for p in v])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
692 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
693 printDBError(error)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
694 if len(set([p.getTrajectoryType() for p in prototypes])) > 1:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
695 print('Different types of prototypes in database ({}).'.format(set([p.getTrajectoryType() for p in prototypes])))
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
696 return prototypes
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 915
diff changeset
697
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
698 def savePOIsToSqlite(filename, gmm, gmmType, gmmId):
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
699 '''Saves a Gaussian mixture model (of class sklearn.mixture.GaussianMixture)
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
700 gmmType is a type of GaussianMixture, learnt either from beginnings or ends of trajectories'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
701 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
702 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
703 if gmmType not in ['beginning', 'end']:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
704 print('Unknown POI type {}. Exiting'.format(gmmType))
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
705 import sys
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
706 sys.exit()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
707 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
708 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (poi_id INTEGER, id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covariance VARCHAR, covariance_type VARCHAR, weight, precisions_cholesky VARCHAR, PRIMARY KEY(poi_id, id))')
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
709 for i in range(gmm.n_components):
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
710 cursor.execute('INSERT INTO gaussians2d VALUES(?,?,?,?,?,?,?,?,?)', (gmmId, i, gmmType, gmm.means_[i][0], gmm.means_[i][1], str(gmm.covariances_[i].tolist()), gmm.covariance_type, gmm.weights_[i], str(gmm.precisions_cholesky_[i].tolist())))
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
711 connection.commit()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
712 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
713 printDBError(error)
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
714
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
715 def savePOIAssignmentsToSqlite(filename, objects):
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
716 'save the od fields of objects'
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
717 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
718 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
719 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
720 cursor.execute('CREATE TABLE IF NOT EXISTS objects_pois (object_id INTEGER, origin_poi_id INTEGER, destination_poi_id INTEGER, PRIMARY KEY(object_id))')
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
721 for o in objects:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
722 cursor.execute('INSERT INTO objects_pois VALUES(?,?,?)', (o.getNum(), o.od[0], o.od[1]))
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
723 connection.commit()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
724 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
725 printDBError(error)
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
726
927
c030f735c594 added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 922
diff changeset
727 def loadPOIsFromSqlite(filename):
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
728 'Loads all 2D Gaussians in the database'
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
729 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
730 from ast import literal_eval
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
731 pois = []
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
732 if Path(filename).is_file():
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
733 with sqlite3.connect(filename) as connection:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
734 cursor = connection.cursor()
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
735 try:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
736 cursor.execute('SELECT * from gaussians2d')
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
737 gmmId = None
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
738 gmm = []
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
739 for row in cursor:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
740 if gmmId is None or row[0] != gmmId:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
741 if len(gmm) > 0:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
742 tmp = mixture.GaussianMixture(len(gmm), covarianceType)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
743 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
744 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
745 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
746 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
747 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
748 pois.append(tmp)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
749 gaussian = {'type': row[2],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
750 'mean': row[3:5],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
751 'covar': array(literal_eval(row[5])),
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
752 'weight': row[7],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
753 'precisions': array(literal_eval(row[8]))}
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
754 gmm = [gaussian]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
755 covarianceType = row[6]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
756 gmmId = row[0]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
757 else:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
758 gmm.append({'type': row[2],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
759 'mean': row[3:5],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
760 'covar': array(literal_eval(row[5])),
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
761 'weight': row[7],
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
762 'precisions': array(literal_eval(row[8]))})
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
763 if len(gmm) > 0:
1200
4356065ed3ca updated simple moving average filter and cleaned tests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
764 tmp = mixture.GaussianMixture(n_components=len(gmm), covariance_type=covarianceType)
1050
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
765 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
766 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
767 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
768 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
769 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
770 pois.append(tmp)
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
771 except sqlite3.OperationalError as error:
9d4a06f49cb8 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1049
diff changeset
772 printDBError(error)
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
773 return pois
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
774
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
775 #########################
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
776 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
777 #########################
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
778
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
779 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
780 ''' prototype dataset is a dictionary with keys== routes, values== prototypes Ids '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
781 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
782 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
783
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
784 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))')
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
785
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
786 for route in prototypes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
787 if prototypes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
788 for i in prototypes[route]:
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
789 cursor.execute('insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)',(i,route[0],route[1],nMatching[route][i]))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
790
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
791 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
792 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
793
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
794 def readPrototypesFromSqlite(filename):
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
795 '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
796 This function loads the prototype file in the database
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
797 It returns a dictionary for prototypes for each route and nMatching
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
798 '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
799 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
800 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
801
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
802 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
803 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
804
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
805 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
806 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
807 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
808 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
809 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
810
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
811 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
812 route=(row[1],row[2])
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
813 if route not in prototypes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
814 prototypes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
815 prototypes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
816 nMatching[row[0]]=row[3]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
817
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
818 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
819 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
820
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
821 def writeLabelsToSqlite(labels, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
822 """ labels is a dictionary with keys: routes, values: prototypes Ids
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
823 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
824 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
825 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
826
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
827 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
828
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
829 for route in labels:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
830 if labels[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
831 for i in labels[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
832 for j in labels[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
833 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
834
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
835 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
836 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
837
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
838 def loadLabelsFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
839 labels = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
840
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
841 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
842 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
843
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
844 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
845 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
846 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
847 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
848 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
849
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
850 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
851 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
852 p=row[3]
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
853 if route not in labels:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
854 labels[route]={}
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
855 if p not in labels[route]:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
856 labels[route][p]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
857 labels[route][p].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
858
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
859 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
860 return labels
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
861
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
862 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
863 """ to match the format of second layer prototypes"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
864 connection = sqlite3.connect(outFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
865 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
866
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
867 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
868
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
869 for route in prototypes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
870 if prototypes[route]!={}:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
871 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
872 if prototypes[route][i]!= []:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
873 for j in prototypes[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
874 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
875
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
876 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
877 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
878
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
879 def loadSpeedPrototypeFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
880 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
881 This function loads the prototypes table in the database of name <filename>.
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
882 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
883 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
884 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
885 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
886 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
887
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
888 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
889 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
890 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
891 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
892 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
893
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
894 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
895 route=(row[2],row[3])
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
896 if route not in prototypes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
897 prototypes[route]={}
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
898 if row[1] not in prototypes[route]:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
899 prototypes[route][row[1]]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
900 prototypes[route][row[1]].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
901 nMatching[row[0]]=row[4]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
902
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
903 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
904 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
905
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
906
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
907 def writeRoutesToSqlite(Routes, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
908 """ This function writes the activity path define by start and end IDs"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
909 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
910 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
911
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
912 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
913
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
914 for route in Routes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
915 if Routes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
916 for i in Routes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
917 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
918
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
919 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
920 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
921
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
922 def loadRoutesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
923 Routes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
924
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
925 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
926 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
927
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
928 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
929 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
930 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
931 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
932 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
933
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
934 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
935 route=(row[1],row[2])
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
936 if route not in Routes:
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
937 Routes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
938 Routes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
939
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
940 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
941 return Routes
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
942
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
943 def setRoutes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
944 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
945 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
946 for obj in objects:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
947 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum()))
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
948 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum()))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
949 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
950 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
951
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
952 #########################
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
953 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
954 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
955
1131
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
956 def loadCSVs(filenames, **kwargs):
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
957 '''Loads all the data from the filenames (eg from glob) and returns a concatenated dataframe'''
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
958 data = read_csv(filenames[0], **kwargs)
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
959 for f in filenames[1:]:
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
960 data = data.append(read_csv(filenames[0], **kwargs))
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
961 return data
3972d85e3b6c work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1095
diff changeset
962
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
963 def saveList(filename, l):
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
964 f = utils.openCheck(filename, 'w')
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
965 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
966 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
967 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
968
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
969 def loadListStrings(filename, commentCharacters = utils.commentChar):
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
970 f = utils.openCheck(filename, 'r')
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
971 result = utils.getLines(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
972 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
973 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
974
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
975 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = utils.commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
976 values = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
977 for l in loadListStrings(filename, commentCharacters):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
978 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
979 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
980 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
981
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
982 def addSectionHeader(propertiesFile, headerName = 'main'):
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
983 '''Add fake section header
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
984
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
985 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
986 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
987 '''
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
988 yield '[{}]\n'.format(headerName)
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
989 for line in propertiesFile:
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
990 yield line
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
991
892
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
992 def loadPemsTraffic(filename):
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
993 '''Loads traffic data downloaded from the http://pems.dot.ca.gov clearinghouse
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
994 into pandas dataframe'''
1095
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
995 data=read_csv(filename, nrows = 3)
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
996 headerNames = ['time', 'station', 'district', 'freeway', 'direction', 'lanetype', 'length', 'nsamples', 'pctobserved', 'totalflow', 'occupancy', 'speed'] # default for 5 min
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
997 nLanes = int((len(data.columns)-len(headerNames))/5)
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
998 for i in range(1, nLanes+1):
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
999 headerNames += ['nsamples{}'.format(i), 'flow{}'.format(i), 'occupancy{}'.format(i), 'speed{}'.format(i), 'pctobserved{}'.format(i)]
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
1000 return read_csv(filename, names = headerNames)
e53c6e87bb3f update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1077
diff changeset
1001
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1002 def generatePDLaneColumn(data):
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1003 data['LANE'] = data['LANE\\LINK\\NO'].astype(str)+'_'+data['LANE\\INDEX'].astype(str)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1004
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1005 def convertTrajectoriesVissimToSqlite(filename):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1006 '''Relies on a system call to sqlite3
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1007 sqlite3 [file.sqlite] < import_fzp.sql'''
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1008 sqlScriptFilename = "import_fzp.sql"
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1009 # create sql file
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1010 out = utils.openCheck(sqlScriptFilename, "w")
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1011 out.write(".separator \";\"\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1012 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1013 ".import "+filename+" curvilinear_positions\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1014 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n")
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1015 out.close()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1016 # system call
1004
75601be6019f work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1003
diff changeset
1017 from subprocess import run
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1018 out = utils.openCheck("err.log", "w")
1004
75601be6019f work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1003
diff changeset
1019 run("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out)
753
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
1020 out.close()
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
1021 shutil.os.remove(sqlScriptFilename)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1022
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1023 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds):
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1024 '''Finds the ids of the objects that go through any of the link in the list linkIds'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1025 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1026 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1027 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1028 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1029 cursor.execute(queryStatement)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1030 return [row[0] for row in cursor]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1031 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1032 printDBError(error)
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1033
759
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1034 def getNObjectsInLinkFromVissimFile(filename, linkIds):
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1035 '''Returns the number of objects that traveled through the link ids'''
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1036 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1037 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1038 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1039 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1040 cursor.execute(queryStatement)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1041 return {row[0]:row[1] for row in cursor}
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1042 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1043 printDBError(error)
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1044
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1045 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1046 '''Reads data from VISSIM .fzp trajectory file
755
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1047 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second)
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1048 for example, there seems to be 10 simulation steps per simulated second in VISSIM,
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1049 so simulationStepsPerTimeUnit should be 10,
527
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
1050 so that all times correspond to the number of the simulation step (and can be stored as integers)
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1051
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1052 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1053 (if the object has no such position, it won't be loaded)
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1054
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1055 Assumed to be sorted over time
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1056 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id'''
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1057 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1058
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1059 if usePandas:
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
1060 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1061 generatePDLaneColumn(data)
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1062 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1063 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1064 data = data[data['TIME']>=warmUpLastInstant]
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1065 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1066 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1067 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1068 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1069 tmp = data[data['NO'] == objNum]
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
1070 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']), nObjects = 1)
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1071 # positions should be rounded to nDecimals decimals only
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1072 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1073 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers:
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1074 return list(objects.values())
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1075 else:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1076 if filename.endswith(".fzp"):
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1077 inputfile = utils.openCheck(filename, quitting = True)
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1078 line = utils.readline(inputfile, '*$')
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1079 while len(line) > 0:#for line in inputfile:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1080 data = line.strip().split(';')
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1081 objNum = int(data[1])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1082 instant = float(data[0])*simulationStepsPerTimeUnit
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1083 s = float(data[4])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1084 y = float(data[5])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1085 lane = data[2]+'_'+data[3]
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1086 if objNum not in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1087 if warmUpLastInstant is None or instant >= warmUpLastInstant:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1088 if objectNumbers is None or len(objects) < objectNumbers:
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
1089 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1090 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1091 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1092 objects[objNum].timeInterval.last = instant
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1093 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1094 line = utils.readline(inputfile, '*$')
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1095 elif filename.endswith(".sqlite"):
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1096 with sqlite3.connect(filename) as connection:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1097 cursor = connection.cursor()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1098 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1099 if objectNumbers is not None:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1100 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1101 queryStatement += ' ORDER BY trajectory_id, t'
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1102 try:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1103 cursor.execute(queryStatement)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1104 for row in cursor:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1105 objNum = row[1]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1106 instant = row[0]*simulationStepsPerTimeUnit
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1107 s = row[4]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1108 y = row[5]
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1109 lane = '{}_{}'.format(row[2], row[3])
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1110 if objNum not in objects:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1111 if warmUpLastInstant is None or instant >= warmUpLastInstant:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1112 if objectNumbers is None or len(objects) < objectNumbers:
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
1113 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1)
938
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1114 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1115 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1116 objects[objNum].timeInterval.last = instant
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1117 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1118 except sqlite3.OperationalError as error:
fbf12382f3f8 replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 935
diff changeset
1119 printDBError(error)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1120 else:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1121 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1122 return list(objects.values())
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1123
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1124 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1125 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1126
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1127 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1128 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1129 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1130 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1131 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1132 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1133 indices = indices | (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1134 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1135 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1136 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1137
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1138 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1139 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1140 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1141
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1142 Vehicles are considered finally stationary
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1143 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1144 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1145 (format as string x_y where x is link index and y is lane index)'''
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1146 if filename.endswith(".fzp"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1147 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1148 if lanes is not None:
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1149 columns += ['LANE\\LINK\\NO', 'LANE\\INDEX']
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1150 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1151 data = selectPDLanes(data, lanes)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1152 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1153
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1154 nStationary = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1155 nVehicles = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1156 for name, group in data.groupby(['NO'], sort = False):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1157 nVehicles += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1158 positions = array(group['POS'])
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1159 diff = positions[1:]-positions[:-1]
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1160 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1161 nStationary += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1162 elif filename.endswith(".sqlite"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1163 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1164 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time?
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1165 pass
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1166 else:
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1167 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1168
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1169 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1170
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1171 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True):
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1172 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1173
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1174 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1175 one checks when the sign of the position difference inverts
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1176 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1177 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1178 (format as string x_y where x is link index and y is lane index)'''
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1179 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1180 data = selectPDLanes(data, lanes)
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1181 data = data.convert_objects(convert_numeric=True)
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1182
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1183 merged = merge(data, data, how='inner', left_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], sort = False)
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1184 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1185
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1186 nCollisions = 0
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1187 for name, group in merged.groupby(['LANE\\LINK\\NO', 'LANE\\INDEX', 'NO_x', 'NO_y']):
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1188 diff = group['POS_x']-group['POS_y']
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1189 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1190 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1191 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1192 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1193 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1194 nCollisions += 1
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1195
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1196 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by??
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1197 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id;
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1198 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1199 return nCollisions
524
1dced8932b08 corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 514
diff changeset
1200
173
319a04ba9033 function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 143
diff changeset
1201 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1202 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1203 and returns the list of Feature objects'''
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1204 objects = []
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1205
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1206 inputfile = utils.openCheck(filename, quitting = True)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1207
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1208 def createObject(numbers):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1209 firstFrameNum = int(numbers[1])
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1210 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1211
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1212 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1213 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1214 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1215 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1216 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1217 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
1041
fc7c0f38e8a6 added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1040
diff changeset
1218 userType = int(numbers[10]), nObjects = 1)
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1219 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1220 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1221 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1222 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1223 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1224 obj.timeHeadways = [float(numbers[17])] # seconds
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1225 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1226 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1227 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1228 return obj
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1229
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1230 numbers = utils.readline(inputfile).strip().split()
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1231 if (len(numbers) > 0):
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1232 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1233
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1234 for line in inputfile:
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1235 numbers = line.strip().split()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1236 if obj.getNum() != int(numbers[0]):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1237 # check and adapt the length to deal with issues in NGSIM data
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1238 if (obj.length() != obj.positions.length()):
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1239 print('length pb with object {} ({},{})'.format(obj.getNum(),obj.length(),obj.positions.length()))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1240 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1241 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1242 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1243 if (nObjects>0) and (len(objects)>=nObjects):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1244 break
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1245 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1246 else:
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1247 obj.laneNums.append(int(numbers[13]))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1248 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542
a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 541
diff changeset
1249 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1250 obj.speeds.append(float(numbers[11]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1251 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1252 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1253 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1254 obj.timeHeadways.append(float(numbers[17]))
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1255
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1256 if (obj.size[0] != float(numbers[8])):
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1257 print('changed length obj {}'.format(obj.getNum()))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1258 if (obj.size[1] != float(numbers[9])):
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1259 print('changed width obj {}'.format(obj.getNum()))
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1260
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1261 inputfile.close()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1262 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1263
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1264 # from https://github.com/kuixu/kitti_object_vis
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1265 def inverse_rigid_trans(Tr):
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1266 """ Inverse a rigid body transform matrix (3x4 as [R|t])
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1267 [R'|-R't; 0|1]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1268 """
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1269 inv_Tr = zeros_like(Tr) # 3x4
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1270 inv_Tr[0:3, 0:3] = transpose(Tr[0:3, 0:3])
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1271 inv_Tr[0:3, 3] = dot(-transpose(Tr[0:3, 0:3]), Tr[0:3, 3])
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1272 return inv_Tr
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1273
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1274 def compute3Dbox(anno_list):
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1275 #### LiDAR coordinate, rotate with y
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1276 '''.DS_Store
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1277 ^ z
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1278 |
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1279 |
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1280 |
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1281 . - - - - - - - > x
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1282 /
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1283 /
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1284 /
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1285 v y
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1286
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1287 '''
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1288 from pykitti.utils import rotz
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1289
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1290 R = rotz(anno_list['heading_3d'])
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1291 l = anno_list['l_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1292 w = anno_list['w_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1293 h = anno_list['h_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1294
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1295 x_3d = anno_list['x_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1296 y_3d = anno_list['y_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1297 z_3d = anno_list['z_3d']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1298
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1299 # # 3d bounding box corners x,y,z as center of the 3d bbox
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1300 # # 0 1 2 3 4 5 6 7
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1301 x_corners = [l/2, l/2, -l/2, -l/2, l/2, l/2, -l/2, -l/2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1302 y_corners = [w/2, -w/2, -w/2, w/2, w/2, -w/2, -w/2, w/2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1303 z_corners = [h/2, h/2, h/2, h/2, -h/2, -h/2, -h/2, -h/2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1304
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1305 # 3d bounding box corners x,y,z as center of the bottom surface of the 3d bbox,
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1306 # the difference occurs at z_corners, as it rotates with z_corners.
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1307 # x_corners = [l/2, l/2, -l/2, -l/2, l/2, l/2, -l/2, -l/2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1308 # y_corners = [w/2, -w/2, -w/2, w/2, w/2, -w/2, -w/2, w/2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1309 # z_corners = [h, h, h, h, 0, 0, 0, 0]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1310 corners_3d = dot(R, vstack([x_corners, y_corners, z_corners]))
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1311 # add the center
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1312 corners_3d[0, :] = corners_3d[0, :] + x_3d
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1313 corners_3d[1, :] = corners_3d[1, :] + y_3d
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1314 corners_3d[2, :] = corners_3d[2, :] + z_3d
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1315
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1316 return transpose(corners_3d)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1317
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1318 def loadKITTICalibration(filename):
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1319 '''Loads KITTI calibration data'''
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1320 calib = {}
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1321 with open(filename, 'r') as f:
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1322 for l in f:
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1323 l = l.rstrip()
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1324 if len(l) == 0:
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1325 continue
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1326 key, value = l.split(' ', 1)
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1327 if ":" in key:
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1328 key = key[:-1]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1329 try:
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1330 calib[key] = array([float(x) for x in value.split()])
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1331 except ValueError as e:
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1332 print(e)
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1333 continue
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1334 #calib['Tr_velo_to_cam'] = calib['Tr_velo_cam']
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1335 #calib['Tr_imu_to_velo'] = calib['Tr_imu_velo']
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1336 calib['Tr_cam_velo'] = inverse_rigid_trans(reshape(calib['Tr_velo_cam'], (3, 4)))
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1337 calib['Tr_velo_imu'] = inverse_rigid_trans(reshape(calib['Tr_imu_velo'], (3, 4)))
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1338
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1339 P = reshape(calib['P2'], (3, 4))
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1340 calib['c_u'] = P[0, 2]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1341 calib['c_v'] = P[1, 2]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1342 calib['f_u'] = P[0, 0]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1343 calib['f_v'] = P[1, 1]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1344 calib['b_x'] = P[0, 3] / (-calib['f_u']) # relative
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1345 calib['b_y'] = P[1, 3] / (-calib['f_v'])
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1346 return calib
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1347
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1348 # from https://github.com/utiasSTARS/pykitti/blob/master/pykitti/utils.py
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1349
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1350
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1351 def loadTrajectoriesFromKITTI(filename, kittiCalibration = None, oxts = None, resultFile = False):
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1352 '''Reads data from KITTI ground truth or output from an object detection and tracking method
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1353
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1354 kittiCalibration is obtained from loading training/testing calibration file for each sequence 00XX.txt
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1355 oxts is obtained using utils.load_oxts_packets_and_poses(['./training/oxts/0001.txt']) from pykitti
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1356
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1357 Ref: https://github.com/pratikac/kitti/blob/master/readme.tracking.txt'''
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1358 from pykitti.utils import roty
1233
d5695e0b59d9 saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1232
diff changeset
1359 from trafficintelligence.cvutils import cartesian2Homogeneous
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1360
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1361 if kittiCalibration is not None:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1362 invR0 = linalg.inv(reshape(kittiCalibration['R_rect'], (3, 3)))
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1363 transCam2Velo = transpose(kittiCalibration['Tr_cam_velo'])
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1364
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1365 header = ['frame', # 0, 1, ..., n
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1366 'trackingid', # -1, 0 , 1, ..., k
1207
36f0d18e1fad work in progress on loading kitti (issue with int)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1205
diff changeset
1367 'usertype', # 'Car', 'Pedestrian', ...
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1368 'truncation', # truncated pixel ratio [0..1]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1369 'occlusion', # 0=visible, 1=partly occluded, 2=fully occluded, 3=unknown
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1370 'alpha', # object observation angle [-pi..pi]
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1371 # extract 2d bounding box in 0-based coordinates
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1372 'xmin', # left
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1373 'ymin', # top
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1374 'xmax', # right
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1375 'ymax', # bottom
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1376 # extract 3d bounding box information
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1377 'h', # box height
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1378 'w', # box width
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1379 'l', # box length (in meters)
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1380 'x', 'y', 'z', # location (x,y,z) in camera coord
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1381 'ry'] # yaw angle (around Y-axis in camera coordinates) [-pi..pi]
1216
8356e15fd691 fix for result files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1215
diff changeset
1382 if resultFile:
8356e15fd691 fix for result files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1215
diff changeset
1383 header.append('score')
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1384 data = read_csv(filename, delimiter = ' ', names = header)
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1385 data = data[data.trackingid > -1]
1202
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1386
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1387 objects = []
1203
7b3384a8e409 second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1202
diff changeset
1388 featureNum = 0
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1389 for objNum in data.trackingid.unique():
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1390 tmp = data[data.trackingid == objNum].sort_values('frame')
1202
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1391 t = moving.Trajectory()#([[float(numbers[6])],[float(numbers[7])]])
1215
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
1392 interval = moving.TimeInterval(int(tmp.frame.min()), int(tmp.frame.max()))
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
1393 userType = tmp.iloc[0].usertype
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
1394 if len(tmp) != interval.length(): #interpolate
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
1395 print(objNum, len(tmp), interval.length())
1b472cddf9b1 updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1213
diff changeset
1396 instants = set(interval).difference(tmp.frame)
1251
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1397 missing = concat([tmp[header[10:]], DataFrame([[t]+[NaN]*(len(header)-10) for t in instants], columns = ['frame']+header[10:])], ignore_index=True).sort_values('frame')
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1398 tmp = missing.interpolate()
1203
7b3384a8e409 second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1202
diff changeset
1399 featureTrajectories = [moving.Trajectory() for i in range(4)]
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1400 for i, r in tmp.iterrows():
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1401 if kittiCalibration is not None:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1402 _, Tr_imu_to_world = oxts[r.frame]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1403 # transfer_matrix = {'P2': reshape(kittiCalibration['P2'], (3, 4)),
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1404 # 'R_rect': kittiCalibration['R_rect'],
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1405 # 'Tr_cam_to_velo': kittiCalibration['Tr_cam_velo'],
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1406 # # transfer_matrix['Tr_velo_to_cam'] = kittiCalibration['Tr_velo_to_cam']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1407 # 'Tr_velo_to_imu': kittiCalibration['Tr_velo_imu'],
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1408 # # transfer_matrix['Tr_imu_to_velo'] = kittiCalibration['Tr_imu_to_velo']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1409 # 'Tr_imu_to_world': Tr_imu2w}
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1410 # 3D object
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1411 # compute rotational matrix around yaw axis
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1412 R = roty(r.ry)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1413
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1414 # 3d bounding box corners
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1415 x_corners = [r.l / 2, r.l / 2, -r.l / 2, -r.l / 2, r.l / 2, r.l / 2, -r.l / 2, -r.l / 2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1416 y_corners = [0, 0, 0, 0, -r.h, -r.h, -r.h, -r.h]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1417 z_corners = [r.w / 2, -r.w / 2, -r.w / 2, r.w / 2, r.w / 2, -r.w / 2, -r.w / 2, r.w / 2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1418 # rotate and translate 3d bounding box
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1419 corners3d = dot(R, vstack([x_corners, y_corners, z_corners]))
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1420 corners3d[0, :] = corners3d[0, :] + r.x#obj.t[0]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1421 corners3d[1, :] = corners3d[1, :] + r.y#obj.t[1]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1422 corners3d[2, :] = corners3d[2, :] + r.z#obj.t[2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1423 # corners3d = transpose(corners3d)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1424 # box3d_pts_3d_velo = calib.project_rect_to_velo(box3d_pts_3d)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1425 refCorners = transpose(dot(invR0, corners3d)) # 8x3 avoid double transpose np.transpose(pts_3d_rect))) in pts_3d_ref = self.project_rect_to_ref(pts_3d_rect)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1426 homRefCorners = cartesian2Homogeneous(refCorners)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1427 veloCorners = dot(homRefCorners, transCam2Velo)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1428 # self.project_ref_to_velo(pts_3d_ref)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1429 # boxes3d_world_single, flag_imu, flag_world = calib_function.project_velo_to_world(bboxes_3d_velo=boxes3d_velo_single, Tr_matrix=transfer_matrix)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1430 #Tr_velo_to_imu = transfer_matrix['Tr_velo_to_imu']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1431 #Tr_imu_to_world = transfer_matrix['Tr_imu_to_world']
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1432 homVeloCorners = cartesian2Homogeneous(veloCorners)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1433 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1434
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1435 homImuCorners = cartesian2Homogeneous(imuCorners)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1436 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1437 else: #LUMPI format
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1438 anno_list_temp = {}
1257
e59a0a475a0a removed bug with unnecessary data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1256
diff changeset
1439 #anno_list_temp['x_2d'] = r.xmin
e59a0a475a0a removed bug with unnecessary data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1256
diff changeset
1440 #anno_list_temp['y_2d'] = r.ymin
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1441 anno_list_temp['l_3d'] = r.l
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1442 anno_list_temp['w_3d'] = r.w
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1443 anno_list_temp['h_3d'] = r.h
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1444 anno_list_temp['x_3d'] = r.x
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1445 anno_list_temp['y_3d'] = r.y
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1446 anno_list_temp['z_3d'] = r.z
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1447 anno_list_temp['heading_3d'] = r.ry
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1448 worldCorners = compute3Dbox(anno_list_temp)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1449 # take first 4 lines of corners, x,y,_ # x0, y0, _ = boxes3d[0]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1450 xCoords = worldCorners[:4,0]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1451 yCoords = worldCorners[:4,1]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1452 t.addPositionXY(xCoords.mean(), yCoords.mean())
1256
56d0195d043e cleaning indicators (no more time interval) and runtimeerror with arccos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1255
diff changeset
1453 for j in range(4):
56d0195d043e cleaning indicators (no more time interval) and runtimeerror with arccos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1255
diff changeset
1454 featureTrajectories[j].addPositionXY(xCoords[j], yCoords[j])
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1455 # check https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1456 if interval.length()>1:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1457 objects.append(moving.MovingObject(num = objNum, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = userType, features = [moving.MovingObject(num = featureNum+i, timeInterval = copy(interval), positions = featureTrajectories[i], velocities = featureTrajectories[i].differentiate(True)) for i in range(4)]))
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1458 featureNum += 4
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1459
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1460 # ego vehicle
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1461 if oxts is not None:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1462 t = moving.Trajectory()
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1463 featureTrajectories = [moving.Trajectory() for i in range(4)]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1464 interval = moving.TimeInterval(0, len(oxts)-1)#int(data.frame.min()), int(data.frame.max()))
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1465 R = roty(pi/2)
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1466 #for frame in interval:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1467 for _, Tr_imu_to_world in oxts:
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1468 l = 4.5 # m
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1469 w = 1.8 # m
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1470 h = 0.
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1471 # 3d bounding box corners
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1472 x_corners = [l / 2, l / 2, -l / 2, -l / 2, l / 2, l / 2, -l / 2, -l / 2]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1473 y_corners = [0, 0, 0, 0, -h, -h, -h, -h]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1474 z_corners = [w / 2, -w / 2, -w / 2, w / 2, w / 2, -w / 2, -w / 2, w / 2]
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1475 # rotate and translate 3d bounding box
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1476 corners3d = dot(R, vstack([x_corners, y_corners, z_corners]))
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1477 corners3d[0, :] = corners3d[0, :]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1478 corners3d[1, :] = corners3d[1, :]
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1479 corners3d[2, :] = corners3d[2, :]
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1480 refCorners = transpose(dot(invR0, corners3d)) # 8x3 avoid double transpose np.transpose(pts_3d_rect))) in pts_3d_ref = self.project_rect_to_ref(pts_3d_rect)
1233
d5695e0b59d9 saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1232
diff changeset
1481 homRefCorners = cartesian2Homogeneous(refCorners)
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1482 veloCorners = dot(homRefCorners, transCam2Velo)
1233
d5695e0b59d9 saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1232
diff changeset
1483 homVeloCorners = cartesian2Homogeneous(veloCorners)
1203
7b3384a8e409 second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1202
diff changeset
1484 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1485
1233
d5695e0b59d9 saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1232
diff changeset
1486 homImuCorners = cartesian2Homogeneous(imuCorners)
1205
3905b393ade0 kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1204
diff changeset
1487 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1488
1202
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1489 # take first 4 lines of corners, x,y,_ # x0, y0, _ = boxes3d[0]
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1490 xCoords = worldCorners[:4,0]
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1491 yCoords = worldCorners[:4,1]
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1492 t.addPositionXY(xCoords.mean(), yCoords.mean())
1203
7b3384a8e409 second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1202
diff changeset
1493 for i in range(4):
7b3384a8e409 second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1202
diff changeset
1494 featureTrajectories[i].addPositionXY(xCoords[i], yCoords[i])
1255
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1495 # check https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c
c0fe55a6b82f added support for LUMPI 3D data format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1254
diff changeset
1496 objects.append(moving.MovingObject(num = max([o.getNum() for o in objects])+1, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = 'Car', features = [moving.MovingObject(num = featureNum+i, timeInterval = copy(interval), positions = featureTrajectories[i], velocities = featureTrajectories[i].differentiate(True)) for i in range(4)]))
1212
af329f3330ba work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1207
diff changeset
1497
1202
059b7282aa09 first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1201
diff changeset
1498 return objects
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1499
1253
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1500 def loadTrajectoriesFromKITTI2D(filename, kittiCalibration = None, oxts = None):
1251
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1501 '''Loads x,y coordinate series
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1502 e.g. obtained by projecting from image to ground world plane using a homography
1253
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1503 Format: frame_id,track_id,usertype,x,y,score
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1504
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1505 if oxts is not None, it is obtained using utils.load_oxts_packets_and_poses(['./training/oxts/0001.txt']) from pykitti
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1506 to generate the movement of the ego vehicle'''
1251
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1507
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1508 header = ['frame','trackingid','usertype','x','y','score']
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1509 data = read_csv(filename, delimiter=' ', names = header)
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1510 objects = []
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1511 for objNum in data.trackingid.unique():
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1512 tmp = data[data.trackingid == objNum].sort_values('frame')
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1513 interval = moving.TimeInterval(int(tmp.frame.min()), int(tmp.frame.max()))
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1514 userType = tmp.iloc[0].usertype
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1515 if len(tmp) != interval.length(): #interpolate
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1516 print(objNum, len(tmp), interval.length())
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1517 instants = set(interval).difference(tmp.frame)
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1518 missing = concat([tmp[['frame', 'x', 'y']], DataFrame([[t, NaN, NaN] for t in instants], columns = ['frame', 'x', 'y'])], ignore_index=True).sort_values('frame')
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1519 tmp = missing.interpolate()
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1520 #print(tmp.info())
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1521 if interval.length()>1:
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1522 t = moving.Trajectory([tmp.x.to_list(), tmp.y.to_list()])
1253
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1523 objects.append(moving.MovingObject(num = objNum, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = userType))
1251
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1524
1253
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1525 # ego vehicle
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1526 if kittiCalibration is not None and oxts is not None:
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1527 invR0 = linalg.inv(reshape(kittiCalibration['R_rect'], (3, 3)))
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1528 transCam2Velo = transpose(kittiCalibration['Tr_cam_velo'])
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1529 t = moving.Trajectory()
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1530 featureTrajectories = [moving.Trajectory() for i in range(4)]
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1531 interval = moving.TimeInterval(0, len(oxts)-1)#int(data.frame.min()), int(data.frame.max()))
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1532 #from pykitti.utils import roty
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1533 from trafficintelligence.cvutils import cartesian2Homogeneous
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1534 for _, Tr_imu_to_world in oxts:
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1535 #R = roty(pi/2)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1536 #corners3d = dot(R, [0,0,0])
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1537 #print(corners3d)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1538 #refCorners = transpose(dot(invR0, corners3d))
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1539 #print(invR0, refCorners) # 000
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1540 #homRefCorners = cartesian2Homogeneous(array([refCorners]))
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1541 veloCorners = transCam2Velo[3, :] # dot(homRefCorners, transCam2Velo)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1542 #print(transCam2Velo, veloCorners) # last column of transCam2Velo
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1543 homVeloCorners = cartesian2Homogeneous(veloCorners)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1544 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1545 #print(kittiCalibration['Tr_velo_imu'], imuCorners)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1546 homImuCorners = cartesian2Homogeneous(imuCorners)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1547 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1548 #print(Tr_imu_to_world, worldCorners)
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1549 xCoords = worldCorners[:4,0]
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1550 yCoords = worldCorners[:4,1]
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1551 t.addPositionXY(xCoords.mean(), yCoords.mean())
ef68d4ba7dae added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1251
diff changeset
1552 objects.append(moving.MovingObject(num = max([o.getNum() for o in objects])+1, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = 'Car'))
1251
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1553 return objects
2b1c8fe8f7e4 added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1250
diff changeset
1554
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1555 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1556 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1557 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1558 if append:
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1559 out = utils.openCheck(outputfile,'a')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1560 else:
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1561 out = utils.openCheck(outputfile,'w')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1562 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1563
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1564 features = loadNgsimFile(inputfile, sequenceNum)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1565 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1566 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1567 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1568
978
184f1dd307f9 corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 976
diff changeset
1569 print(nObjectsPerType)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1570
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1571 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1572
895
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1573 def loadPinholeCameraModel(filename, tanalystFormat = True):
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1574 '''Loads the data from a file containing the camera parameters
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1575 (pinhole camera model, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1576 and returns a dictionary'''
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1577 if tanalystFormat:
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1578 f = utils.openCheck(filename, quitting = True)
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1579 content = utils.getLines(f)
895
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1580 cameraData = {}
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1581 for l in content:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1582 tmp = l.split(':')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1583 cameraData[tmp[0]] = float(tmp[1].strip().replace(',','.'))
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1584 return cameraData
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1585 else:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1586 print('Unknown camera model (not tanalyst format')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1587 return None
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1588
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1589 def savePositionsToCsv(f, obj):
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1590 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1591 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1592 curvilinearPositions = obj.getCurvilinearPositions()
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1593 for i in range(int(obj.length())):
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1594 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1595 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1596 if curvilinearPositions is not None:
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1597 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1598 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1599 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1600
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1601 def saveTrajectoriesToCsv(filename, objects):
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1602 f = utils.openCheck(filename, 'w')
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1603 for i,obj in enumerate(objects):
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1604 savePositionsToCsv(f, obj)
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1605 f.close()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1606
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1607
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1608 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1609 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1610 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1611
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1612 class ClassifierParameters(VideoFilenameAddable):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1613 'Class for the parameters of object classifiers'
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1614 def loadConfigFile(self, filename):
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1615 from configparser import ConfigParser
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1616
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1617 config = ConfigParser()
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1618 config.read_file(addSectionHeader(utils.openCheck(filename)))
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1619
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1620 parentPath = Path(filename).parent
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1621 self.sectionHeader = config.sections()[0]
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1622
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1623 self.pedBikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'pbv-svm-filename'))
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1624 self.bikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'bv-svm-filename'))
1241
ab4c72b9475c work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1239
diff changeset
1625 self.dlFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'dl-filename'))
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1626 self.percentIncreaseCrop = config.getfloat(self.sectionHeader, 'percent-increase-crop')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1627 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
1189
ccab20f85710 changed to better parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1182
diff changeset
1628 x = config.getint(self.sectionHeader, 'hog-rescale-size')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1629 self.hogRescaleSize = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1630 self.hogNOrientations = config.getint(self.sectionHeader, 'hog-norientations')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1631 x = config.getint(self.sectionHeader, 'hog-npixels-cell')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1632 self.hogNPixelsPerCell = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1633 x = config.getint(self.sectionHeader, 'hog-ncells-block')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1634 self.hogNCellsPerBlock = (x, x)
893
ff92801e5c54 updated hog to scikit-image 0.13 (needed to add a block_norm attribute in classifier.cfg)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 892
diff changeset
1635 self.hogBlockNorm = config.get(self.sectionHeader, 'hog-block-norm')
1242
4cd8ace3552f major update for classification, allowing the use of neural network classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1241
diff changeset
1636 self.confidence = config.getfloat(self.sectionHeader, 'confidence')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1637
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1638 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1639 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
1022
b7689372c0ec renamed quantile to centile
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1021
diff changeset
1640 self.speedAggregationCentile = config.getint(self.sectionHeader, 'speed-aggregation-centile')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1641 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
899
1466a63dd1cf added a new classification parameter
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 895
diff changeset
1642 self.maxPercentUnknown = config.getfloat(self.sectionHeader, 'max-prop-unknown-appearance')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1643 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1644 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1645 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1646 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1647 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1648 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1649 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1650 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1651
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1652 def __init__(self, filename = None):
1058
16575ca4537d work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1056
diff changeset
1653 self.configFilename = filename
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
1654 if filename is not None and Path(filename).is_file():
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1655 self.loadConfigFile(filename)
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1656 else:
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1657 print('Configuration filename {} could not be loaded.'.format(filename))
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1658
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1659 def convertToFrames(self, frameRate, speedRatio = 3.6):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1660 '''Converts parameters with a relationship to time in 'native' frame time
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1661 speedRatio is the conversion from the speed unit in the config file
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1662 to the distance per second
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1663
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1664 ie param(config file) = speedRatio x fps x param(used in program)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1665 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1666 denominator = frameRate*speedRatio
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1667 #denominator2 = denominator**2
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1668 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1669 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1670 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1671 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1672 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1673 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1674 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1675 # special case for the lognormal distribution
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1676 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
854
33d296984dd8 rework and more info on speed probabilities for classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 852
diff changeset
1677 #self.scaleCyclistSpeed = self.scaleCyclistSpeed # no modification of scale
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1678
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1679
664
455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 660
diff changeset
1680 class ProcessParameters(VideoFilenameAddable):
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1681 '''Class for all parameters controlling data processing: input,
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1682 method parameters, etc. for tracking and safety
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1683
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1684 Note: framerate is already taken into account'''
1075
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1685
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1686 def loadConfigFile(self, filename):
1052
1748c02f9ac3 Modifying storage.py [configloaders]
Wendlasida
parents: 1041
diff changeset
1687 from configparser import ConfigParser
1075
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1688
1053
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1689 config = ConfigParser({ 'acceleration-bound' : '3',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1690 'min-velocity-cosine' : '0.8',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1691 'ndisplacements' : '3',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1692 'max-nfeatures' : '1000',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1693 'feature-quality' : '0.0812219538558',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1694 'min-feature-distanceklt' : '3.54964337411',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1695 'block-size' : '7',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1696 'use-harris-detector' : '0',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1697 'k' : '0.04',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1698 'window-size' : '6',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1699 'pyramid-level' : '5',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1700 'min-tracking-error' : '0.183328975142',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1701 'max-number-iterations' : '20',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1702 'feature-flag' : '0',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1703 'min-feature-eig-threshold' : '1e-4',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1704 'min-feature-time' : '15',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1705 'min-feature-displacement' : '0.05',
60cc87e824c4 Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents: 1052
diff changeset
1706 'tracker-reload-time' : '10'}, strict=False)
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1707 config.read_file(addSectionHeader(utils.openCheck(filename)))
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1708 parentPath = Path(filename).parent
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1709 self.sectionHeader = config.sections()[0]
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1710 # Tracking/display parameters
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1711 self.videoFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'video-filename'))
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1712 self.databaseFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'database-filename'))
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1713 self.homographyFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'homography-filename'))
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
1714 if Path(self.homographyFilename).is_file():
882
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1715 self.homography = loadtxt(self.homographyFilename)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1716 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1717 self.homography = None
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1718 self.intrinsicCameraFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'intrinsic-camera-filename'))
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
1719 if Path(self.intrinsicCameraFilename).is_file():
882
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1720 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1721 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1722 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1723 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1724 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1725 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
1249
2aa56b101041 added mask functionality for dltrack
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1246
diff changeset
1726 self.maskFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'mask-filename'))
2aa56b101041 added mask functionality for dltrack
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1246
diff changeset
1727
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1728 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1729 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1730 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
948
584b9405e494 added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 943
diff changeset
1731
1021
16932cefabc1 work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1004
diff changeset
1732 self.classifierFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'classifier-filename'))
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1733
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1734 # Safety parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1735 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
976
7f5cbdc107c5 corrected bug in converting time horizon to frames for internal computations (results for TTC, pPET and PET will be in frames)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 948
diff changeset
1736 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1737 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1738 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1739 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1740 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1741 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1742 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1743 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1744 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1745 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1746 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
943
b1e8453c207c work on motion prediction using motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 938
diff changeset
1747 self.constantSpeedPrototypePrediction = config.getboolean(self.sectionHeader, 'constant-speed')
948
584b9405e494 added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 943
diff changeset
1748 self.maxLcssDistance = config.getfloat(self.sectionHeader, 'max-lcss-distance')
584b9405e494 added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 943
diff changeset
1749 self.lcssMetric = config.get(self.sectionHeader, 'lcss-metric')
584b9405e494 added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 943
diff changeset
1750 self.minLcssSimilarity = config.getfloat(self.sectionHeader, 'min-lcss-similarity')
1052
1748c02f9ac3 Modifying storage.py [configloaders]
Wendlasida
parents: 1041
diff changeset
1751
1075
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1752 # Tracking parameters
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1753 self.accelerationBound = config.getint(self.sectionHeader, 'acceleration-bound')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1754 self.minVelocityCosine = config.getfloat(self.sectionHeader, 'min-velocity-cosine')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1755 self.ndisplacements = config.getint(self.sectionHeader, 'ndisplacements')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1756 self.maxNFeatures = config.getint(self.sectionHeader, 'max-nfeatures')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1757 self.minFeatureDistanceKLT = config.getfloat(self.sectionHeader, 'min-feature-distanceklt')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1758 self.featureQuality = config.getfloat(self.sectionHeader, 'feature-quality')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1759 self.blockSize = config.getint(self.sectionHeader, 'block-size')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1760 self.useHarrisDetector = config.getboolean(self.sectionHeader, 'use-harris-detector')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1761 self.k = config.getfloat(self.sectionHeader, 'k')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1762 self.winSize = config.getint(self.sectionHeader, 'window-size')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1763 self.pyramidLevel = config.getint(self.sectionHeader, 'pyramid-level')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1764 self.maxNumberTrackingIterations = config.getint(self.sectionHeader, 'max-number-iterations')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1765 self.minTrackingError = config.getfloat(self.sectionHeader, 'min-tracking-error')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1766 self.featureFlags = config.getboolean(self.sectionHeader, 'feature-flag')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1767 self.minFeatureEigThreshold = config.getfloat(self.sectionHeader, 'min-feature-eig-threshold')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1768 self.minFeatureTime = config.getint(self.sectionHeader, 'min-feature-time')
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1769 self.minFeatureDisplacement = config.getfloat(self.sectionHeader, 'min-feature-displacement')
1250
77fbd0e2ba7d dltrack works with moving average filtering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1249
diff changeset
1770 self.smoothingHalfWidth = config.getint(self.sectionHeader, 'smoothing-halfwidth')
1232
83ca1493d55c minor work / verif saving works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1216
diff changeset
1771 #self.updateTimer = config.getint(self.sectionHeader, 'tracker-reload-time')
1075
67144f26609e Updates crop
Wendlasida
parents: 1054
diff changeset
1772
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1773
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1774 def __init__(self, filename = None):
1058
16575ca4537d work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1056
diff changeset
1775 self.configFilename = filename
1065
d4d052a05337 added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1058
diff changeset
1776 if filename is not None and Path(filename).is_file():
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1777 self.loadConfigFile(filename)
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1778 else:
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1779 print('Configuration filename {} could not be loaded.'.format(filename))
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1780
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1781 def processVideoArguments(args):
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1782 '''Loads information from configuration file
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1783 then checks what was passed on the command line
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1784 for override (eg video filename and database filename'''
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1785 if args.configFilename is not None: # consider there is a configuration file
1177
aa88acf06876 rewrote object concatenation method, cleaner
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1143
diff changeset
1786 parentPath = Path(args.configFilename).parent
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1787 params = ProcessParameters(args.configFilename)
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1788 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1789 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1790 if params.homography is not None:
1246
2397de73770d dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1245
diff changeset
1791 homography = params.homography
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1792 invHomography = linalg.inv(params.homography)
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1793 else:
1246
2397de73770d dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1245
diff changeset
1794 homography = None
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1795 invHomography = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1796 intrinsicCameraMatrix = params.intrinsicCameraMatrix
935
0e63a918a1ca updated classify-objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 927
diff changeset
1797 distortionCoefficients = array(params.distortionCoefficients)
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1798 undistortedImageMultiplication = params.undistortedImageMultiplication
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1799 undistort = params.undistort
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1800 firstFrameNum = params.firstFrameNum
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1801 else:
1177
aa88acf06876 rewrote object concatenation method, cleaner
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1143
diff changeset
1802 params = None
1246
2397de73770d dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1245
diff changeset
1803 homography = None
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1804 invHomography = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1805 undistort = False
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1806 intrinsicCameraMatrix = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1807 distortionCoefficients = []
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1808 undistortedImageMultiplication = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1809 undistort = False
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1810 firstFrameNum = 0
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1811
1245
371c718e57d7 interface updates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1242
diff changeset
1812 # override video and database filenames if present on command line (path is relative to command line location)
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1813 if args.videoFilename is not None:
1026
73b124160911 more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1022
diff changeset
1814 videoFilename = args.videoFilename
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1815 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1816 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1817 if args.databaseFilename is not None:
1026
73b124160911 more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1022
diff changeset
1818 databaseFilename = args.databaseFilename
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1819 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1820 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1821
1246
2397de73770d dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1245
diff changeset
1822 return params, videoFilename, databaseFilename, homography, invHomography, intrinsicCameraMatrix, distortionCoefficients, undistortedImageMultiplication, undistort, firstFrameNum
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1823
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1824 # deprecated
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
1825 class SceneParameters(object):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1826 def __init__(self, config, sectionName):
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1827 from configparser import NoOptionError
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1828 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1829 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1830 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1831 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1832 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1833 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1834 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1835 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1836 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1837 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1838 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1839 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1840 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1841 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1842 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1843
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1844 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1845 def loadConfigFile(filename):
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 979
diff changeset
1846 from configparser import ConfigParser
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1847 config = ConfigParser()
1030
aafbc0bab925 moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
1848 config.readfp(utils.openCheck(filename))
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1849 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1850 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1851 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1852 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1853
382
ba813f148ade development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 377
diff changeset
1854
1201
28bf2420c412 work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1189
diff changeset
1855 if __name__ == '__main__':
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1856 import doctest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1857 import unittest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1858 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1859 unittest.TextTestRunner().run(suite)
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1860 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1861 # #doctest.testfile("example.txt")