Mercurial Hosting > traffic-intelligence
annotate trafficintelligence/storage.py @ 1253:ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
author | Nicolas Saunier <nicolas.saunier@polymtl.ca> |
---|---|
date | Mon, 25 Mar 2024 17:05:20 -0400 |
parents | 2b1c8fe8f7e4 |
children | a477ad82ab66 |
rev | line source |
---|---|
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1 #! /usr/bin/env python |
208
d9855499fc88
Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents:
204
diff
changeset
|
2 # -*- coding: utf-8 -*- |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
3 '''Various utilities to save and load data''' |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
4 |
1003
75af46516b2b
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
5 from pathlib import Path |
1028
cc5cb04b04b0
major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1026
diff
changeset
|
6 import shutil |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
7 from copy import copy |
417
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
8 import sqlite3, logging |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
9 |
1215
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
10 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt, floor as npfloor, ceil as npceil, linalg, int32, int64, reshape, dot, vstack, transpose, ones, zeros_like, pi, NaN |
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
11 from pandas import read_csv, merge, concat, DataFrame |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
12 |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
13 from trafficintelligence import utils, moving, events, indicators |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
14 from trafficintelligence.base import VideoFilenameAddable |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
15 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
16 |
50 | 17 ngsimUserTypes = {'twowheels':1, |
204
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
18 'car':2, |
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
19 'truck':3} |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
20 |
908
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
21 tableNames = {'feature':'positions', |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
22 'object': 'objects', |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
23 'objectfeatures': 'positions'} |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
24 |
1049
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
25 sqlite3.register_adapter(int64, lambda val: int(val)) |
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
26 sqlite3.register_adapter(int32, lambda val: int(val)) |
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
27 |
259
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
28 ######################### |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
29 # Sqlite |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
30 ######################### |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
31 |
491
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
32 # utils |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
33 def printDBError(error): |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
34 print('DB Error: {}'.format(error)) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
35 |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
36 def dropTables(connection, tableNames): |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
37 'deletes the table with names in tableNames' |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
38 try: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
39 cursor = connection.cursor() |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
40 for tableName in tableNames: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
41 cursor.execute('DROP TABLE IF EXISTS '+tableName) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
42 except sqlite3.OperationalError as error: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
43 printDBError(error) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
44 |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
45 def deleteFromSqlite(filename, dataType): |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
46 'Deletes (drops) some tables in the filename depending on type of data' |
1003
75af46516b2b
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
47 if Path(filename).is_file(): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
48 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
49 if dataType == 'object': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
50 dropTables(connection, ['objects', 'objects_features']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
51 elif dataType == 'interaction': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
52 dropTables(connection, ['interactions', 'indicators']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
53 elif dataType == 'bb': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
54 dropTables(connection, ['bounding_boxes']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
55 elif dataType == 'pois': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
56 dropTables(connection, ['gaussians2d', 'objects_pois']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
57 elif dataType == 'prototype': |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
58 dropTables(connection, ['prototypes', 'objects_prototypes', 'features_prototypes']) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
59 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
60 print('Unknown data type {} to delete from database'.format(dataType)) |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
61 else: |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
62 print('{} does not exist'.format(filename)) |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
63 |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
64 def tableExists(connection, tableName): |
740
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
65 'indicates if the table exists in the database' |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
66 try: |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
67 cursor = connection.cursor() |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
68 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'') |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
69 return cursor.fetchone()[0] == 1 |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
70 except sqlite3.OperationalError as error: |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
71 printDBError(error) |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
72 |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
73 def tableNames(filename): |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
74 'Lists the names of the tables in the SQLite file' |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
75 if Path(filename).is_file(): |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
76 with sqlite3.connect(filename) as connection: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
77 try: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
78 cursor = connection.cursor() |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
79 cursor.execute('SELECT name FROM sqlite_master WHERE type = \'table\'') |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
80 return [row[0] for row in cursor] |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
81 except sqlite3.OperationalError as error: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
82 printDBError(error) |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
83 return [] |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
84 |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
85 def createTrajectoryTable(cursor, tableName): |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
86 if tableName.endswith('positions') or tableName.endswith('velocities'): |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
87 cursor.execute("CREATE TABLE IF NOT EXISTS "+tableName+" (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))") |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
88 else: |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
89 print('Unallowed name {} for trajectory table'.format(tableName)) |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
90 |
879 | 91 def createObjectsTable(cursor): |
92 cursor.execute("CREATE TABLE IF NOT EXISTS objects (object_id INTEGER, road_user_type INTEGER, n_objects INTEGER, PRIMARY KEY(object_id))") | |
93 | |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
94 def createAssignmentTable(cursor, objectType1, objectType2, objectIdColumnName1, objectIdColumnName2): |
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
95 cursor.execute("CREATE TABLE IF NOT EXISTS "+objectType1+"s_"+objectType2+"s ("+objectIdColumnName1+" INTEGER, "+objectIdColumnName2+" INTEGER, PRIMARY KEY("+objectIdColumnName1+","+objectIdColumnName2+"))") |
913
1cd878812529
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
910
diff
changeset
|
96 |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
97 def createObjectsFeaturesTable(cursor): |
913
1cd878812529
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
910
diff
changeset
|
98 cursor.execute("CREATE TABLE IF NOT EXISTS objects_features (object_id INTEGER, trajectory_id INTEGER, PRIMARY KEY(object_id, trajectory_id))") |
879 | 99 |
100 | |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
101 def createCurvilinearTrajectoryTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
102 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))") |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
103 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
104 def createFeatureCorrespondenceTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
105 cursor.execute('CREATE TABLE IF NOT EXISTS feature_correspondences (trajectory_id INTEGER, source_dbname VARCHAR, db_trajectory_id INTEGER, PRIMARY KEY(trajectory_id))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
106 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
107 def createInteractionTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
108 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
109 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
110 def createIndicatorTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
111 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
112 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
113 def insertTrajectoryQuery(tableName): |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
114 return "INSERT INTO "+tableName+" VALUES (?,?,?,?)" |
831
a8ff35e6fb43
forgotten change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
830
diff
changeset
|
115 |
879 | 116 def insertObjectQuery(): |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
117 return "INSERT INTO objects VALUES (?,?,?)" |
879 | 118 |
119 def insertObjectFeatureQuery(): | |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
120 return "INSERT INTO objects_features VALUES (?,?)" |
879 | 121 |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
122 def createIndex(connection, tableName, columnName, unique = False): |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
123 '''Creates an index for the column in the table |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
124 I will make querying with a condition on this column faster''' |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
125 try: |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
126 cursor = connection.cursor() |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
127 s = "CREATE " |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
128 if unique: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
129 s += "UNIQUE " |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
130 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")") |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
131 connection.commit() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
132 #connection.close() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
133 except sqlite3.OperationalError as error: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
134 printDBError(error) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
135 |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
136 def getNumberRowsTable(connection, tableName, columnName = None): |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
137 '''Returns the number of rows for the table |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
138 If columnName is not None, means we want the number of distinct values for that column |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
139 (otherwise, we can just count(*))''' |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
140 try: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
141 cursor = connection.cursor() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
142 if columnName is None: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
143 cursor.execute("SELECT COUNT(*) from "+tableName) |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
144 else: |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
145 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
146 return cursor.fetchone()[0] |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
147 except sqlite3.OperationalError as error: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
148 printDBError(error) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
149 |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
150 def getMinMax(connection, tableName, columnName, minmax): |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
151 '''Returns max/min or both for given column in table |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
152 minmax must be string max, min or minmax''' |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
153 try: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
154 cursor = connection.cursor() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
155 if minmax == 'min' or minmax == 'max': |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
156 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
157 elif minmax == 'minmax': |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
158 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
159 else: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
160 print("Argument minmax unknown: {}".format(minmax)) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
161 return cursor.fetchone()[0] |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
162 except sqlite3.OperationalError as error: |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
163 printDBError(error) |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
164 |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
165 def getObjectCriteria(objectNumbers): |
585
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
166 if objectNumbers is None: |
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
167 query = '' |
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
168 elif type(objectNumbers) == int: |
852
45a53542e046
updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
850
diff
changeset
|
169 query = '<= {0}'.format(objectNumbers-1) |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
170 elif type(objectNumbers) == list: |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
171 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')' |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
172 else: |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
173 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers))) |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
174 query = '' |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
175 return query |
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
176 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
177 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None, timeStep = None): |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
178 '''Loads trajectories (in the general sense) from the given table |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
179 can be positions or velocities |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
180 |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
181 returns a moving object''' |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
182 cursor = connection.cursor() |
204
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
183 |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
184 try: |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
185 objectCriteria = getObjectCriteria(objectNumbers) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
186 queryStatement = None |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
187 if trajectoryType == 'feature': |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
188 queryStatement = 'SELECT * from '+tableName |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
189 if objectNumbers is not None and timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
190 queryStatement += ' WHERE trajectory_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep) |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
191 elif objectNumbers is not None: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
192 queryStatement += ' WHERE trajectory_id '+objectCriteria |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
193 elif timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
194 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep) |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
195 queryStatement += ' ORDER BY trajectory_id, frame_number' |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
196 elif trajectoryType == 'object': |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
197 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
198 if objectNumbers is not None: |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
199 queryStatement += ' AND OF.object_id '+objectCriteria |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
200 if timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
201 queryStatement += ' AND P.frame_number%{} = 0'.format(timeStep) |
712
21aeadcfbabb
added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
711
diff
changeset
|
202 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number' |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
203 elif trajectoryType in ['bbtop', 'bbbottom']: |
587
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
204 if trajectoryType == 'bbtop': |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
205 corner = 'top_left' |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
206 elif trajectoryType == 'bbbottom': |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
207 corner = 'bottom_right' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
208 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
209 if objectNumbers is not None and timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
210 queryStatement += ' WHERE object_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep) |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
211 elif objectNumbers is not None: |
712
21aeadcfbabb
added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
711
diff
changeset
|
212 queryStatement += ' WHERE object_id '+objectCriteria |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
213 elif timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
214 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep) |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
215 queryStatement += ' ORDER BY object_id, frame_number' |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
216 else: |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
217 print('Unknown trajectory type {}'.format(trajectoryType)) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
218 if queryStatement is not None: |
417
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
219 cursor.execute(queryStatement) |
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
220 logging.debug(queryStatement) |
342
4d69486869a5
work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
341
diff
changeset
|
221 except sqlite3.OperationalError as error: |
491
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
222 printDBError(error) |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
223 return [] |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
224 |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
225 objId = -1 |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
226 obj = None |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
227 objects = [] |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
228 for row in cursor: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
229 if row[0] != objId: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
230 objId = row[0] |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
231 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())): |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
232 objects.append(obj) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
233 elif obj is not None: |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
234 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length())) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
235 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]])) |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
236 else: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
237 obj.timeInterval.last = row[1] |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
238 obj.positions.addPositionXY(row[2],row[3]) |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
239 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
240 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())): |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
241 objects.append(obj) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
242 elif obj is not None: |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
243 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length())) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
244 |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
245 return objects |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
246 |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
247 def loadObjectAttributesFromTable(cursor, objectNumbers, loadNObjects = False): |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
248 objectCriteria = getObjectCriteria(objectNumbers) |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
249 queryStatement = 'SELECT object_id, road_user_type' |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
250 if loadNObjects: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
251 queryStatement += ', n_objects' |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
252 queryStatement += ' FROM objects' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
253 if objectNumbers is not None: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
254 queryStatement += ' WHERE object_id '+objectCriteria |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
255 cursor.execute(queryStatement) |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
256 attributes = {} |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
257 if loadNObjects: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
258 for row in cursor: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
259 attributes[row[0]] = row[1:] |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
260 else: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
261 for row in cursor: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
262 attributes[row[0]] = row[1] |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
263 return attributes |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
264 |
1044
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
265 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False, timeStep = None, nLongestFeaturesPerObject = None): |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
266 '''Loads the trajectories (in the general sense, |
1044
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
267 either features, objects (feature groups), longest features per object, or bounding box series) |
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
268 types are only feature or object |
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
269 if object, features can be loaded with withFeatures or nLongestObjectFeatures used to select the n longest features |
1040
20799ac9524e
integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1039
diff
changeset
|
270 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
271 The number loaded is either the first objectNumbers objects, |
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
272 or the indices in objectNumbers from the database''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
273 objects = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
274 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
275 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
276 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
277 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers, timeStep) |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
278 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
279 if len(objectVelocities) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
280 for o,v in zip(objects, objectVelocities): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
281 if o.getNum() == v.getNum(): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
282 o.velocities = v.positions |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
283 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
284 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
285 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum())) |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
286 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
287 if trajectoryType == 'object': |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
288 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
289 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
290 # attribute feature numbers to objects |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
291 queryStatement = 'SELECT trajectory_id, object_id FROM objects_features' |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
292 if objectNumbers is not None: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
293 queryStatement += ' WHERE object_id '+getObjectCriteria(objectNumbers) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
294 queryStatement += ' ORDER BY object_id' # order is important to group all features per object |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
295 logging.debug(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
296 cursor.execute(queryStatement) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
297 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
298 featureNumbers = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
299 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
300 objId = row[1] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
301 if objId not in featureNumbers: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
302 featureNumbers[objId] = [row[0]] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
303 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
304 featureNumbers[objId].append(row[0]) |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
305 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
306 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
307 obj.featureNumbers = featureNumbers[obj.getNum()] |
685
94b291a5f933
several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
680
diff
changeset
|
308 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
309 # load userType |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
310 attributes = loadObjectAttributesFromTable(cursor, objectNumbers, True) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
311 for obj in objects: |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
312 userType, nObjects = attributes[obj.getNum()] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
313 obj.setUserType(userType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
314 obj.setNObjects(nObjects) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
315 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
316 # add features |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
317 if withFeatures: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
318 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
319 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', obj.featureNumbers, timeStep = timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
320 elif nLongestFeaturesPerObject is not None: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
321 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
322 queryStatement = 'SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions WHERE trajectory_id '+getObjectCriteria(obj.featureNumbers)+' GROUP BY trajectory_id ORDER BY length DESC' |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
323 logging.debug(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
324 cursor.execute(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
325 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', [row[0] for i,row in enumerate(cursor) if i<nLongestFeaturesPerObject], timeStep = timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
326 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
327 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
328 printDBError(error) |
1178
ee3eaf902b83
minor improvement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1177
diff
changeset
|
329 else: |
ee3eaf902b83
minor improvement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1177
diff
changeset
|
330 print('Impossible to load from non-existing file '+filename) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
331 return objects |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
332 |
910
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
333 def loadObjectFeatureFrameNumbers(filename, objectNumbers = None): |
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
334 'Loads the feature frame numbers for each object' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
335 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
336 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
337 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
338 queryStatement = 'SELECT OF.object_id, TL.trajectory_id, TL.length FROM (SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions GROUP BY trajectory_id) TL, objects_features OF WHERE TL.trajectory_id = OF.trajectory_id' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
339 if objectNumbers is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
340 queryStatement += ' AND object_id '+getObjectCriteria(objectNumbers) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
341 queryStatement += ' ORDER BY OF.object_id, TL.length DESC' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
342 logging.debug(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
343 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
344 objectFeatureNumbers = {} |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
345 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
346 objId = row[0] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
347 if objId in objectFeatureNumbers: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
348 objectFeatureNumbers[objId].append(row[1]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
349 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
350 objectFeatureNumbers[objId] = [row[1]] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
351 return objectFeatureNumbers |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
352 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
353 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
354 return None |
910
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
355 |
780
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
356 def addCurvilinearTrajectoriesFromSqlite(filename, objects): |
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
357 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane) |
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
358 from a database to an existing MovingObject dict (indexed by each objects's num)''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
359 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
360 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
361 |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
362 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
363 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
364 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
365 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
366 return [] |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
367 |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
368 missingObjectNumbers = [] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
369 objNum = None |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
370 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
371 if objNum != row[0]: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
372 objNum = row[0] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
373 if objNum in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
374 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
375 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
376 missingObjectNumbers.append(objNum) |
784
30bd0f2223b7
tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
783
diff
changeset
|
377 if objNum in objects: |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
378 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
379 if len(missingObjectNumbers) > 0: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
380 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers)) |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
381 |
1039
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
382 def saveTrajectoriesToTable(connection, objects, trajectoryType): |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
383 'Saves trajectories in table tableName' |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
384 cursor = connection.cursor() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
385 # Parse feature and/or object structure and commit to DB |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
386 if(trajectoryType == 'feature' or trajectoryType == 'object'): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
387 # Extract features from objects |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
388 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
389 features = [] |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
390 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
391 if obj.hasFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
392 features += obj.getFeatures() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
393 if len(features) == 0: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
394 print('Warning, objects have no features') # todo save centroid trajectories? |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
395 elif trajectoryType == 'feature': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
396 features = objects |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
397 # Setup feature queries |
1039
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
398 createTrajectoryTable(cursor, "positions") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
399 createTrajectoryTable(cursor, "velocities") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
400 positionQuery = insertTrajectoryQuery("positions") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
401 velocityQuery = insertTrajectoryQuery("velocities") |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
402 # Setup object queries |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
403 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
404 createObjectsTable(cursor) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
405 createObjectsFeaturesTable(cursor) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
406 objectQuery = insertObjectQuery() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
407 objectFeatureQuery = insertObjectFeatureQuery() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
408 for feature in features: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
409 num = feature.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
410 frameNum = feature.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
411 for p in feature.getPositions(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
412 cursor.execute(positionQuery, (num, frameNum, p.x, p.y)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
413 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
414 velocities = feature.getVelocities() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
415 if velocities is not None: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
416 frameNum = feature.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
417 for v in velocities[:-1]: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
418 cursor.execute(velocityQuery, (num, frameNum, v.x, v.y)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
419 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
420 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
421 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
422 if obj.hasFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
423 for feature in obj.getFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
424 featureNum = feature.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
425 cursor.execute(objectFeatureQuery, (obj.getNum(), featureNum)) |
1052 | 426 cursor.execute(objectQuery, (obj.getNum(), obj.getUserType(), obj.nObjects if hasattr(obj, 'nObjects') and obj.nObjects is not None else 1)) |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
427 # Parse curvilinear position structure |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
428 elif(trajectoryType == 'curvilinear'): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
429 createCurvilinearTrajectoryTable(cursor) |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
430 curvilinearQuery = "INSERT INTO curvilinear_positions VALUES (?,?,?,?,?)" |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
431 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
432 num = obj.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
433 frameNum = obj.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
434 for p in obj.getCurvilinearPositions(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
435 cursor.execute(curvilinearQuery, (num, frameNum, p[0], p[1], p[2])) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
436 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
437 else: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
438 print('Unknown trajectory type {}'.format(trajectoryType)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
439 connection.commit() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
440 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
441 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType): |
829 | 442 '''Writes features, ie the trajectory positions (and velocities if exist) |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
443 with their instants to a specified sqlite file |
780
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
444 Either feature positions (and velocities if they exist) |
879 | 445 or curvilinear positions will be saved at a time''' |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
446 |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
447 with sqlite3.connect(outputFilename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
448 try: |
1040
20799ac9524e
integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1039
diff
changeset
|
449 saveTrajectoriesToTable(connection, objects, trajectoryType) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
450 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
451 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
452 |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
453 def setRoadUserTypes(filename, objects): |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
454 '''Saves the user types of the objects in the sqlite database stored in filename |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
455 The objects should exist in the objects table''' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
456 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
457 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
458 for obj in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
459 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum())) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
460 connection.commit() |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
461 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
462 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None, timeStep = None): |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
463 '''Loads bounding box moving object from an SQLite |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
464 (format of SQLite output by the ground truth annotation tool |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
465 or Urban Tracker |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
466 |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
467 Load descriptions?''' |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
468 objects = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
469 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
470 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
471 if objectType == 'bb': |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
472 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
473 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
474 userTypes = loadObjectAttributesFromTable(connection.cursor(), objectNumbers) # string format is same as object |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
475 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
476 for t, b in zip(topCorners, bottomCorners): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
477 num = t.getNum() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
478 if t.getNum() == b.getNum(): |
1133
c4d9c270f999
modification for performance computation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1131
diff
changeset
|
479 annotation = moving.BBMovingObject(num, t, b, t.getTimeInterval(), userTypes[num]) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
480 objects.append(annotation) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
481 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
482 print ('Unknown type of bounding box {}'.format(objectType)) |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
483 return objects |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
484 |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
485 def saveInteraction(cursor, interaction): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
486 roadUserNumbers = list(interaction.getRoadUserNumbers()) |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
487 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant())) |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
488 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
489 def saveInteractionsToSqlite(filename, interactions): |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
490 'Saves the interactions in the table' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
491 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
492 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
493 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
494 createInteractionTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
495 for inter in interactions: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
496 saveInteraction(cursor, inter) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
497 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
498 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
499 connection.commit() |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
500 |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
501 def saveIndicator(cursor, interactionNum, indicator): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
502 for instant in indicator.getTimeInterval(): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
503 if indicator[instant]: |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
504 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant])) |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
505 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
506 def saveIndicatorsToSqlite(filename, interactions, indicatorNames = events.Interaction.indicatorNames): |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
507 'Saves the indicator values in the table' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
508 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
509 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
510 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
511 createInteractionTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
512 createIndicatorTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
513 for inter in interactions: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
514 saveInteraction(cursor, inter) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
515 for indicatorName in indicatorNames: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
516 indicator = inter.getIndicator(indicatorName) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
517 if indicator is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
518 saveIndicator(cursor, inter.getNum(), indicator) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
519 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
520 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
521 connection.commit() |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
522 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
523 def loadInteractionsFromSqlite(filename): |
482
f6415f012640
adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
417
diff
changeset
|
524 '''Loads interaction and their indicators |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
525 |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
526 TODO choose the interactions to load''' |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
527 interactions = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
528 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
529 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
530 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
531 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
532 cursor.execute('SELECT INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
533 interactionNum = -1 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
534 indicatorTypeNum = -1 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
535 tmpIndicators = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
536 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
537 if row[0] != interactionNum: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
538 interactionNum = row[0] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
539 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2])) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
540 interactions[-1].indicators = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
541 if indicatorTypeNum != row[5] or row[0] != interactionNum: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
542 indicatorTypeNum = row[5] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
543 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
544 indicatorValues = {row[6]:row[7]} |
1182
0e5d37b0b9ff
bug corrections
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1178
diff
changeset
|
545 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.mostSevereIsMinIndicators) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
546 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
547 indicatorValues[row[6]] = row[7] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
548 interactions[-1].indicators[indicatorName].timeInterval.last = row[6] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
549 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
550 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
551 return [] |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
552 return interactions |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
553 # load first and last object instants |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
554 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id |
342
4d69486869a5
work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
341
diff
changeset
|
555 |
390
12be4a0cb9aa
sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
382
diff
changeset
|
556 def createBoundingBoxTable(filename, invHomography = None): |
482
f6415f012640
adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
417
diff
changeset
|
557 '''Create the table to store the object bounding boxes in image space |
390
12be4a0cb9aa
sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
382
diff
changeset
|
558 ''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
559 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
560 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
561 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
562 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
563 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from ' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
564 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from ' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
565 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+ |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
566 'GROUP BY object_id, frame_number') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
567 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
568 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
569 connection.commit() |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
570 |
586
ff4f0ce46ca6
modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
585
diff
changeset
|
571 def loadBoundingBoxTableForDisplay(filename): |
647
458890c0437c
cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
646
diff
changeset
|
572 '''Loads bounding boxes from bounding_boxes table for display over trajectories''' |
393
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
573 boundingBoxes = {} # list of bounding boxes for each instant |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
574 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
575 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
576 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
577 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
578 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
579 result = cursor.fetchall() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
580 if len(result) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
581 cursor.execute('SELECT * FROM bounding_boxes') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
582 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
583 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
584 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
585 printDBError(error) |
393
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
586 return boundingBoxes |
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
587 |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
588 ######################### |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
589 # saving and loading for scene interpretation: POIs and Prototypes |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
590 ######################### |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
591 |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
592 def savePrototypesToSqlite(filename, prototypes): |
1033
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
593 '''save the prototypes (a prototype is defined by a filename, a number (id) and type)''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
594 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
595 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
596 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
597 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), nmatchings INTEGER, PRIMARY KEY (prototype_filename, prototype_id, trajectory_type))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
598 for p in prototypes: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
599 cursor.execute('INSERT INTO prototypes VALUES(?,?,?,?)', (p.getFilename(), p.getNum(), p.getTrajectoryType(), p.getNMatchings())) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
600 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
601 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
602 connection.commit() |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
603 |
1033
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
604 def setPrototypeMatchingsInSqlite(filename, prototypes): |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
605 '''updates the prototype matchings''' |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
606 with sqlite3.connect(filename) as connection: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
607 cursor = connection.cursor() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
608 try: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
609 for p in prototypes: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
610 if p.getNMatchings() is None: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
611 nMatchings = 'NULL' |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
612 else: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
613 nMatchings = p.getNMatchings() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
614 cursor.execute('UPDATE prototypes SET nmatchings = {} WHERE prototype_filename = \"{}\" AND prototype_id = {} AND trajectory_type = \"{}\"'.format(nMatchings, p.getFilename(), p.getNum(), p.getTrajectoryType())) |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
615 except sqlite3.OperationalError as error: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
616 printDBError(error) |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
617 connection.commit() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
618 |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
619 def prototypeAssignmentNames(objectType): |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
620 tableName = objectType+'s_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
621 if objectType == 'feature': |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
622 #tableName = 'features_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
623 objectIdColumnName = 'trajectory_id' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
624 elif objectType == 'object': |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
625 #tableName = 'objects_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
626 objectIdColumnName = 'object_id' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
627 return tableName, objectIdColumnName |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
628 |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
629 def savePrototypeAssignmentsToSqlite(filename, objectNumbers, objectType, labels, prototypes): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
630 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
631 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
632 try: |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
633 tableName, objectIdColumnName = prototypeAssignmentNames(objectType) |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
634 cursor.execute('CREATE TABLE IF NOT EXISTS '+tableName+' ('+objectIdColumnName+' INTEGER, prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), PRIMARY KEY('+objectIdColumnName+', prototype_filename, prototype_id, trajectory_type))') |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
635 for objNum, label in zip(objectNumbers, labels): |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
636 if label >=0: |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
637 proto = prototypes[label] |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
638 cursor.execute('INSERT INTO '+tableName+' VALUES(?,?,?,?)', (objNum, proto.getFilename(), proto.getNum(), proto.getTrajectoryType())) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
639 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
640 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
641 connection.commit() |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
642 |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
643 def loadPrototypeAssignmentsFromSqlite(filename, objectType): |
1239
31173c4699d2
minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1233
diff
changeset
|
644 '''Loads the assignments between prototypes |
31173c4699d2
minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1233
diff
changeset
|
645 Returns a dictionary with prototypes as keys and a list of |
31173c4699d2
minor comment
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1233
diff
changeset
|
646 the objects/feature nums assigned to each''' |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
647 prototypeAssignments = {} |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
648 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
649 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
650 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
651 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
652 tableName, objectIdColumnName = prototypeAssignmentNames(objectType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
653 cursor.execute('SELECT * FROM '+tableName) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
654 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
655 p = moving.Prototype(row[1], row[2], row[3]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
656 if p in prototypeAssignments: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
657 prototypeAssignments[p].append(row[0]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
658 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
659 prototypeAssignments[p] = [row[0]] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
660 return prototypeAssignments |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
661 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
662 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
663 return prototypeAssignments |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
664 |
920
499154254f37
improved prototype loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
919
diff
changeset
|
665 def loadPrototypesFromSqlite(filename, withTrajectories = True): |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
666 'Loads prototype ids and matchings (if stored)' |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
667 prototypes = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
668 if Path(filename).is_file(): |
1143
8ac52ebff5f7
avoid using chdir
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1141
diff
changeset
|
669 parentPath = Path(filename).resolve().parent |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
670 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
671 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
672 objects = [] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
673 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
674 cursor.execute('SELECT * FROM prototypes') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
675 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
676 prototypes.append(moving.Prototype(row[0], row[1], row[2], row[3])) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
677 if withTrajectories: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
678 for p in prototypes: |
1143
8ac52ebff5f7
avoid using chdir
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1141
diff
changeset
|
679 p.setMovingObject(loadTrajectoriesFromSqlite(str(parentPath/p.getFilename()), p.getTrajectoryType(), [p.getNum()])[0]) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
680 # loadingInformation = {} # complicated slightly optimized |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
681 # for p in prototypes: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
682 # dbfn = p.getFilename() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
683 # trajType = p.getTrajectoryType() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
684 # if (dbfn, trajType) in loadingInformation: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
685 # loadingInformation[(dbfn, trajType)].append(p) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
686 # else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
687 # loadingInformation[(dbfn, trajType)] = [p] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
688 # for k, v in loadingInformation.iteritems(): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
689 # objects += loadTrajectoriesFromSqlite(k[0], k[1], [p.getNum() for p in v]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
690 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
691 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
692 if len(set([p.getTrajectoryType() for p in prototypes])) > 1: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
693 print('Different types of prototypes in database ({}).'.format(set([p.getTrajectoryType() for p in prototypes]))) |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
694 return prototypes |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
695 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
696 def savePOIsToSqlite(filename, gmm, gmmType, gmmId): |
871
6db83beb5350
work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
862
diff
changeset
|
697 '''Saves a Gaussian mixture model (of class sklearn.mixture.GaussianMixture) |
6db83beb5350
work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
862
diff
changeset
|
698 gmmType is a type of GaussianMixture, learnt either from beginnings or ends of trajectories''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
699 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
700 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
701 if gmmType not in ['beginning', 'end']: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
702 print('Unknown POI type {}. Exiting'.format(gmmType)) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
703 import sys |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
704 sys.exit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
705 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
706 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (poi_id INTEGER, id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covariance VARCHAR, covariance_type VARCHAR, weight, precisions_cholesky VARCHAR, PRIMARY KEY(poi_id, id))') |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
707 for i in range(gmm.n_components): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
708 cursor.execute('INSERT INTO gaussians2d VALUES(?,?,?,?,?,?,?,?,?)', (gmmId, i, gmmType, gmm.means_[i][0], gmm.means_[i][1], str(gmm.covariances_[i].tolist()), gmm.covariance_type, gmm.weights_[i], str(gmm.precisions_cholesky_[i].tolist()))) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
709 connection.commit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
710 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
711 printDBError(error) |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
712 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
713 def savePOIAssignmentsToSqlite(filename, objects): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
714 'save the od fields of objects' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
715 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
716 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
717 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
718 cursor.execute('CREATE TABLE IF NOT EXISTS objects_pois (object_id INTEGER, origin_poi_id INTEGER, destination_poi_id INTEGER, PRIMARY KEY(object_id))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
719 for o in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
720 cursor.execute('INSERT INTO objects_pois VALUES(?,?,?)', (o.getNum(), o.od[0], o.od[1])) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
721 connection.commit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
722 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
723 printDBError(error) |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
724 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
725 def loadPOIsFromSqlite(filename): |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
726 'Loads all 2D Gaussians in the database' |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
727 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields |
872
c70adaeeddf5
solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
871
diff
changeset
|
728 from ast import literal_eval |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
729 pois = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
730 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
731 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
732 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
733 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
734 cursor.execute('SELECT * from gaussians2d') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
735 gmmId = None |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
736 gmm = [] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
737 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
738 if gmmId is None or row[0] != gmmId: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
739 if len(gmm) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
740 tmp = mixture.GaussianMixture(len(gmm), covarianceType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
741 tmp.means_ = array([gaussian['mean'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
742 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
743 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
744 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
745 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
746 pois.append(tmp) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
747 gaussian = {'type': row[2], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
748 'mean': row[3:5], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
749 'covar': array(literal_eval(row[5])), |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
750 'weight': row[7], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
751 'precisions': array(literal_eval(row[8]))} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
752 gmm = [gaussian] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
753 covarianceType = row[6] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
754 gmmId = row[0] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
755 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
756 gmm.append({'type': row[2], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
757 'mean': row[3:5], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
758 'covar': array(literal_eval(row[5])), |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
759 'weight': row[7], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
760 'precisions': array(literal_eval(row[8]))}) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
761 if len(gmm) > 0: |
1200
4356065ed3ca
updated simple moving average filter and cleaned tests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
762 tmp = mixture.GaussianMixture(n_components=len(gmm), covariance_type=covarianceType) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
763 tmp.means_ = array([gaussian['mean'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
764 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
765 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
766 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
767 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
768 pois.append(tmp) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
769 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
770 printDBError(error) |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
771 return pois |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
772 |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
773 ######################### |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
774 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
775 ######################### |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
776 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
777 def writePrototypesToSqlite(prototypes,nMatching, outputFilename): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
778 ''' prototype dataset is a dictionary with keys== routes, values== prototypes Ids ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
779 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
780 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
781 |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
782 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))') |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
783 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
784 for route in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
785 if prototypes[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
786 for i in prototypes[route]: |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
787 cursor.execute('insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)',(i,route[0],route[1],nMatching[route][i])) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
788 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
789 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
790 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
791 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
792 def readPrototypesFromSqlite(filename): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
793 ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
794 This function loads the prototype file in the database |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
795 It returns a dictionary for prototypes for each route and nMatching |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
796 ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
797 prototypes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
798 nMatching={} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
799 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
800 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
801 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
802 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
803 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
804 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
805 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
806 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
807 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
808 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
809 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
810 route=(row[1],row[2]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
811 if route not in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
812 prototypes[route]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
813 prototypes[route].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
814 nMatching[row[0]]=row[3] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
815 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
816 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
817 return prototypes,nMatching |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
818 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
819 def writeLabelsToSqlite(labels, outputFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
820 """ labels is a dictionary with keys: routes, values: prototypes Ids |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
821 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
822 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
823 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
824 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
825 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
826 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
827 for route in labels: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
828 if labels[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
829 for i in labels[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
830 for j in labels[route][i]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
831 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i)) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
832 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
833 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
834 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
835 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
836 def loadLabelsFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
837 labels = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
838 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
839 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
840 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
841 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
842 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
843 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
844 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
845 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
846 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
847 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
848 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
849 route=(row[1],row[2]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
850 p=row[3] |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
851 if route not in labels: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
852 labels[route]={} |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
853 if p not in labels[route]: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
854 labels[route][p]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
855 labels[route][p].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
856 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
857 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
858 return labels |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
859 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
860 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
861 """ to match the format of second layer prototypes""" |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
862 connection = sqlite3.connect(outFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
863 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
864 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
865 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
866 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
867 for route in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
868 if prototypes[route]!={}: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
869 for i in prototypes[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
870 if prototypes[route][i]!= []: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
871 for j in prototypes[route][i]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
872 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j])) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
873 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
874 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
875 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
876 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
877 def loadSpeedPrototypeFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
878 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
879 This function loads the prototypes table in the database of name <filename>. |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
880 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
881 prototypes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
882 nMatching={} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
883 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
884 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
885 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
886 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
887 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
888 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
889 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
890 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
891 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
892 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
893 route=(row[2],row[3]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
894 if route not in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
895 prototypes[route]={} |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
896 if row[1] not in prototypes[route]: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
897 prototypes[route][row[1]]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
898 prototypes[route][row[1]].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
899 nMatching[row[0]]=row[4] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
900 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
901 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
902 return prototypes,nMatching |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
903 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
904 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
905 def writeRoutesToSqlite(Routes, outputFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
906 """ This function writes the activity path define by start and end IDs""" |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
907 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
908 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
909 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
910 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
911 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
912 for route in Routes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
913 if Routes[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
914 for i in Routes[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
915 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1])) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
916 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
917 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
918 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
919 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
920 def loadRoutesFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
921 Routes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
922 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
923 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
924 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
925 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
926 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
927 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
928 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
929 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
930 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
931 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
932 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
933 route=(row[1],row[2]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
934 if route not in Routes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
935 Routes[route]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
936 Routes[route].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
937 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
938 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
939 return Routes |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
940 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
941 def setRoutes(filename, objects): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
942 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
943 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
944 for obj in objects: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
945 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum())) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
946 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum())) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
947 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
948 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
949 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
950 ######################### |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
951 # txt files |
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
952 ######################### |
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
953 |
1131
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
954 def loadCSVs(filenames, **kwargs): |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
955 '''Loads all the data from the filenames (eg from glob) and returns a concatenated dataframe''' |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
956 data = read_csv(filenames[0], **kwargs) |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
957 for f in filenames[1:]: |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
958 data = data.append(read_csv(filenames[0], **kwargs)) |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
959 return data |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
960 |
829 | 961 def saveList(filename, l): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
962 f = utils.openCheck(filename, 'w') |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
963 for x in l: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
964 f.write('{}\n'.format(x)) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
965 f.close() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
966 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
967 def loadListStrings(filename, commentCharacters = utils.commentChar): |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
968 f = utils.openCheck(filename, 'r') |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
969 result = utils.getLines(f, commentCharacters) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
970 f.close() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
971 return result |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
972 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
973 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = utils.commentChar): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
974 values = [] |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
975 for l in loadListStrings(filename, commentCharacters): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
976 if l.startswith(option): |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
977 values.append(l.split(delimiterChar)[1].strip()) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
978 return values |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
979 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
980 def addSectionHeader(propertiesFile, headerName = 'main'): |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
981 '''Add fake section header |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
982 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
983 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
984 use read_file in Python 3.2+ |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
985 ''' |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
986 yield '[{}]\n'.format(headerName) |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
987 for line in propertiesFile: |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
988 yield line |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
989 |
892
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
990 def loadPemsTraffic(filename): |
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
991 '''Loads traffic data downloaded from the http://pems.dot.ca.gov clearinghouse |
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
992 into pandas dataframe''' |
1095
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
993 data=read_csv(filename, nrows = 3) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
994 headerNames = ['time', 'station', 'district', 'freeway', 'direction', 'lanetype', 'length', 'nsamples', 'pctobserved', 'totalflow', 'occupancy', 'speed'] # default for 5 min |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
995 nLanes = int((len(data.columns)-len(headerNames))/5) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
996 for i in range(1, nLanes+1): |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
997 headerNames += ['nsamples{}'.format(i), 'flow{}'.format(i), 'occupancy{}'.format(i), 'speed{}'.format(i), 'pctobserved{}'.format(i)] |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
998 return read_csv(filename, names = headerNames) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
999 |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1000 def generatePDLaneColumn(data): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1001 data['LANE'] = data['LANE\\LINK\\NO'].astype(str)+'_'+data['LANE\\INDEX'].astype(str) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1002 |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1003 def convertTrajectoriesVissimToSqlite(filename): |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1004 '''Relies on a system call to sqlite3 |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1005 sqlite3 [file.sqlite] < import_fzp.sql''' |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1006 sqlScriptFilename = "import_fzp.sql" |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1007 # create sql file |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1008 out = utils.openCheck(sqlScriptFilename, "w") |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1009 out.write(".separator \";\"\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1010 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1011 ".import "+filename+" curvilinear_positions\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1012 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n") |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1013 out.close() |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1014 # system call |
1004
75601be6019f
work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1003
diff
changeset
|
1015 from subprocess import run |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1016 out = utils.openCheck("err.log", "w") |
1004
75601be6019f
work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1003
diff
changeset
|
1017 run("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out) |
753
3d48e34db846
switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
752
diff
changeset
|
1018 out.close() |
3d48e34db846
switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
752
diff
changeset
|
1019 shutil.os.remove(sqlScriptFilename) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1020 |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1021 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds): |
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1022 '''Finds the ids of the objects that go through any of the link in the list linkIds''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1023 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1024 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1025 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1026 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1027 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1028 return [row[0] for row in cursor] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1029 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1030 printDBError(error) |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1031 |
759
a05b70f307dd
added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
756
diff
changeset
|
1032 def getNObjectsInLinkFromVissimFile(filename, linkIds): |
a05b70f307dd
added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
756
diff
changeset
|
1033 '''Returns the number of objects that traveled through the link ids''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1034 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1035 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1036 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1037 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1038 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1039 return {row[0]:row[1] for row in cursor} |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1040 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1041 printDBError(error) |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1042 |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1043 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True): |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1044 '''Reads data from VISSIM .fzp trajectory file |
755
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1045 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second) |
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1046 for example, there seems to be 10 simulation steps per simulated second in VISSIM, |
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1047 so simulationStepsPerTimeUnit should be 10, |
527
37830a831818
loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
526
diff
changeset
|
1048 so that all times correspond to the number of the simulation step (and can be stored as integers) |
660
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1049 |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1050 Objects positions will be considered only after warmUpLastInstant |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1051 (if the object has no such position, it won't be loaded) |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1052 |
752
14963a9c3b09
debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
751
diff
changeset
|
1053 Assumed to be sorted over time |
14963a9c3b09
debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
751
diff
changeset
|
1054 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id''' |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1055 objects = {} # dictionary of objects index by their id |
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1056 |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1057 if usePandas: |
718
2cd245cb780d
added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
714
diff
changeset
|
1058 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1059 generatePDLaneColumn(data) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1060 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit |
660
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1061 if warmUpLastInstant is not None: |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1062 data = data[data['TIME']>=warmUpLastInstant] |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1063 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False) |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1064 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax}) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1065 for row_index, row in instants.iterrows(): |
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1066 objNum = int(row['NO']) |
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1067 tmp = data[data['NO'] == objNum] |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1068 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']), nObjects = 1) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1069 # positions should be rounded to nDecimals decimals only |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1070 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist()) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1071 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers: |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1072 return list(objects.values()) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1073 else: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1074 if filename.endswith(".fzp"): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1075 inputfile = utils.openCheck(filename, quitting = True) |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1076 line = utils.readline(inputfile, '*$') |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1077 while len(line) > 0:#for line in inputfile: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1078 data = line.strip().split(';') |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1079 objNum = int(data[1]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1080 instant = float(data[0])*simulationStepsPerTimeUnit |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1081 s = float(data[4]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1082 y = float(data[5]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1083 lane = data[2]+'_'+data[3] |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1084 if objNum not in objects: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1085 if warmUpLastInstant is None or instant >= warmUpLastInstant: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1086 if objectNumbers is None or len(objects) < objectNumbers: |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1087 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1088 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1089 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1090 objects[objNum].timeInterval.last = instant |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1091 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane) |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1092 line = utils.readline(inputfile, '*$') |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1093 elif filename.endswith(".sqlite"): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1094 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1095 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1096 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1097 if objectNumbers is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1098 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1099 queryStatement += ' ORDER BY trajectory_id, t' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1100 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1101 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1102 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1103 objNum = row[1] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1104 instant = row[0]*simulationStepsPerTimeUnit |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1105 s = row[4] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1106 y = row[5] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1107 lane = '{}_{}'.format(row[2], row[3]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1108 if objNum not in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1109 if warmUpLastInstant is None or instant >= warmUpLastInstant: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1110 if objectNumbers is None or len(objects) < objectNumbers: |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1111 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1112 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1113 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1114 objects[objNum].timeInterval.last = instant |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1115 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1116 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1117 printDBError(error) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1118 else: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1119 print("File type of "+filename+" not supported (only .sqlite and .fzp files)") |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1120 return list(objects.values()) |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1121 |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1122 def selectPDLanes(data, lanes = None): |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1123 '''Selects the subset of data for the right lanes |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1124 |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1125 Lane format is a string 'x_y' where x is link index and y is lane index''' |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1126 if lanes is not None: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1127 if 'LANE' not in data.columns: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1128 generatePDLaneColumn(data) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1129 indices = (data['LANE'] == lanes[0]) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1130 for l in lanes[1:]: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1131 indices = indices | (data['LANE'] == l) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1132 return data[indices] |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1133 else: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1134 return data |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1135 |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1136 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7): |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1137 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1138 and the total number of vehicles |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1139 |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1140 Vehicles are considered finally stationary |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1141 if more than proportionStationaryTime of their total time |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1142 If lanes is not None, only the data for the selected lanes will be provided |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1143 (format as string x_y where x is link index and y is lane index)''' |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1144 if filename.endswith(".fzp"): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1145 columns = ['NO', '$VEHICLE:SIMSEC', 'POS'] |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1146 if lanes is not None: |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1147 columns += ['LANE\\LINK\\NO', 'LANE\\INDEX'] |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1148 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1149 data = selectPDLanes(data, lanes) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1150 data.sort(['$VEHICLE:SIMSEC'], inplace = True) |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1151 |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1152 nStationary = 0 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1153 nVehicles = 0 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1154 for name, group in data.groupby(['NO'], sort = False): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1155 nVehicles += 1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1156 positions = array(group['POS']) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1157 diff = positions[1:]-positions[:-1] |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1158 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1159 nStationary += 1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1160 elif filename.endswith(".sqlite"): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1161 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1162 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time? |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1163 pass |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1164 else: |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1165 print("File type of "+filename+" not supported (only .sqlite and .fzp files)") |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1166 |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1167 return nStationary, nVehicles |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1168 |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1169 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True): |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1170 '''Counts the number of collisions per lane in a VISSIM trajectory file |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1171 |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1172 To distinguish between cars passing and collision, |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1173 one checks when the sign of the position difference inverts |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1174 (if the time are closer than collisionTimeDifference) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1175 If lanes is not None, only the data for the selected lanes will be provided |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1176 (format as string x_y where x is link index and y is lane index)''' |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1177 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1178 data = selectPDLanes(data, lanes) |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1179 data = data.convert_objects(convert_numeric=True) |
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1180 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1181 merged = merge(data, data, how='inner', left_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], sort = False) |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1182 merged = merged[merged['NO_x']>merged['NO_y']] |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1183 |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1184 nCollisions = 0 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1185 for name, group in merged.groupby(['LANE\\LINK\\NO', 'LANE\\INDEX', 'NO_x', 'NO_y']): |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1186 diff = group['POS_x']-group['POS_y'] |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1187 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any |
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1188 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0: |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1189 xidx = diff[diff < 0].argmax() |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1190 yidx = diff[diff > 0].argmin() |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1191 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference: |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1192 nCollisions += 1 |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1193 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1194 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by?? |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1195 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id; |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1196 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0 |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1197 return nCollisions |
524
1dced8932b08
corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
514
diff
changeset
|
1198 |
173
319a04ba9033
function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
143
diff
changeset
|
1199 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1200 '''Reads data from the trajectory data provided by NGSIM project |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1201 and returns the list of Feature objects''' |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1202 objects = [] |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1203 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1204 inputfile = utils.openCheck(filename, quitting = True) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1205 |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1206 def createObject(numbers): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1207 firstFrameNum = int(numbers[1]) |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1208 # do the geometry and usertype |
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1209 |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1210 firstFrameNum = int(numbers[1]) |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1211 lastFrameNum = firstFrameNum+int(numbers[2])-1 |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1212 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1) |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1213 obj = moving.MovingObject(num = int(numbers[0]), |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1214 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum), |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1215 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]), |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1216 userType = int(numbers[10]), nObjects = 1) |
78
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1217 obj.userType = int(numbers[10]) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1218 obj.laneNums = [int(numbers[13])] |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1219 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1220 obj.followingVehicles = [int(numbers[15])] # following vehicle (after) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1221 obj.spaceHeadways = [float(numbers[16])] # feet |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1222 obj.timeHeadways = [float(numbers[17])] # seconds |
327
42f2b46ec210
added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
326
diff
changeset
|
1223 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1224 obj.speeds = [float(numbers[11])] |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1225 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1226 return obj |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1227 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1228 numbers = utils.readline(inputfile).strip().split() |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1229 if (len(numbers) > 0): |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1230 obj = createObject(numbers) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1231 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1232 for line in inputfile: |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1233 numbers = line.strip().split() |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
1234 if obj.getNum() != int(numbers[0]): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1235 # check and adapt the length to deal with issues in NGSIM data |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1236 if (obj.length() != obj.positions.length()): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1237 print('length pb with object {} ({},{})'.format(obj.getNum(),obj.length(),obj.positions.length())) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1238 obj.last = obj.getFirstInstant()+obj.positions.length()-1 |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1239 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ? |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1240 objects.append(obj) |
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1241 if (nObjects>0) and (len(objects)>=nObjects): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1242 break |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1243 obj = createObject(numbers) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1244 else: |
327
42f2b46ec210
added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
326
diff
changeset
|
1245 obj.laneNums.append(int(numbers[13])) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1246 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7])) |
542
a3add9f751ef
added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
541
diff
changeset
|
1247 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1]) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1248 obj.speeds.append(float(numbers[11])) |
78
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1249 obj.precedingVehicles.append(int(numbers[14])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1250 obj.followingVehicles.append(int(numbers[15])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1251 obj.spaceHeadways.append(float(numbers[16])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1252 obj.timeHeadways.append(float(numbers[17])) |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1253 |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1254 if (obj.size[0] != float(numbers[8])): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1255 print('changed length obj {}'.format(obj.getNum())) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1256 if (obj.size[1] != float(numbers[9])): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1257 print('changed width obj {}'.format(obj.getNum())) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1258 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1259 inputfile.close() |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1260 return objects |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1261 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1262 # from https://github.com/kuixu/kitti_object_vis |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1263 def inverse_rigid_trans(Tr): |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1264 """ Inverse a rigid body transform matrix (3x4 as [R|t]) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1265 [R'|-R't; 0|1] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1266 """ |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1267 inv_Tr = zeros_like(Tr) # 3x4 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1268 inv_Tr[0:3, 0:3] = transpose(Tr[0:3, 0:3]) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1269 inv_Tr[0:3, 3] = dot(-transpose(Tr[0:3, 0:3]), Tr[0:3, 3]) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1270 return inv_Tr |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1271 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1272 def loadKITTICalibration(filename): |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1273 '''Loads KITTI calibration data''' |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1274 calib = {} |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1275 with open(filename, 'r') as f: |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1276 for l in f: |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1277 l = l.rstrip() |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1278 if len(l) == 0: |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1279 continue |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1280 key, value = l.split(' ', 1) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1281 if ":" in key: |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1282 key = key[:-1] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1283 try: |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1284 calib[key] = array([float(x) for x in value.split()]) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1285 except ValueError as e: |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1286 print(e) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1287 continue |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1288 #calib['Tr_velo_to_cam'] = calib['Tr_velo_cam'] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1289 #calib['Tr_imu_to_velo'] = calib['Tr_imu_velo'] |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1290 calib['Tr_cam_velo'] = inverse_rigid_trans(reshape(calib['Tr_velo_cam'], (3, 4))) |
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1291 calib['Tr_velo_imu'] = inverse_rigid_trans(reshape(calib['Tr_imu_velo'], (3, 4))) |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1292 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1293 P = reshape(calib['P2'], (3, 4)) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1294 calib['c_u'] = P[0, 2] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1295 calib['c_v'] = P[1, 2] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1296 calib['f_u'] = P[0, 0] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1297 calib['f_v'] = P[1, 1] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1298 calib['b_x'] = P[0, 3] / (-calib['f_u']) # relative |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1299 calib['b_y'] = P[1, 3] / (-calib['f_v']) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1300 return calib |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1301 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1302 # from https://github.com/utiasSTARS/pykitti/blob/master/pykitti/utils.py |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1303 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1304 |
1216
8356e15fd691
fix for result files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1215
diff
changeset
|
1305 def loadTrajectoriesFromKITTI(filename, kittiCalibration, oxts, resultFile = False): |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1306 '''Reads data from KITTI ground truth or output from an object detection and tracking method |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1307 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1308 kittiCalibration is obtained from loading training/testing calibration file for each sequence 00XX.txt |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1309 oxts is obtained using utils.load_oxts_packets_and_poses(['./training/oxts/0001.txt']) from pykitti |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1310 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1311 Ref: https://github.com/pratikac/kitti/blob/master/readme.tracking.txt''' |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1312 from pykitti.utils import roty |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1313 from trafficintelligence.cvutils import cartesian2Homogeneous |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1314 |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1315 invR0 = linalg.inv(reshape(kittiCalibration['R_rect'], (3, 3))) |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1316 transCam2Velo = transpose(kittiCalibration['Tr_cam_velo']) |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1317 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1318 header = ['frame', # 0, 1, ..., n |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1319 'trackingid', # -1, 0 , 1, ..., k |
1207
36f0d18e1fad
work in progress on loading kitti (issue with int)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1205
diff
changeset
|
1320 'usertype', # 'Car', 'Pedestrian', ... |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1321 'truncation', # truncated pixel ratio [0..1] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1322 'occlusion', # 0=visible, 1=partly occluded, 2=fully occluded, 3=unknown |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1323 'alpha', # object observation angle [-pi..pi] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1324 # extract 2d bounding box in 0-based coordinates |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1325 'xmin', # left |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1326 'ymin', # top |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1327 'xmax', # right |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1328 'ymax', # bottom |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1329 # extract 3d bounding box information |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1330 'h', # box height |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1331 'w', # box width |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1332 'l', # box length (in meters) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1333 'x', 'y', 'z', # location (x,y,z) in camera coord |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1334 'ry'] # yaw angle (around Y-axis in camera coordinates) [-pi..pi] |
1216
8356e15fd691
fix for result files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1215
diff
changeset
|
1335 if resultFile: |
8356e15fd691
fix for result files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1215
diff
changeset
|
1336 header.append('score') |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1337 data = read_csv(filename, delimiter = ' ', names = header) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1338 data = data[data.trackingid > -1] |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1339 |
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1340 objects = [] |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1341 featureNum = 0 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1342 for objNum in data.trackingid.unique(): |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1343 tmp = data[data.trackingid == objNum].sort_values('frame') |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1344 t = moving.Trajectory()#([[float(numbers[6])],[float(numbers[7])]]) |
1215
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1345 interval = moving.TimeInterval(int(tmp.frame.min()), int(tmp.frame.max())) |
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1346 userType = tmp.iloc[0].usertype |
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1347 if len(tmp) != interval.length(): #interpolate |
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1348 print(objNum, len(tmp), interval.length()) |
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1349 instants = set(interval).difference(tmp.frame) |
1251
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1350 missing = concat([tmp[header[10:]], DataFrame([[t]+[NaN]*(len(header)-10) for t in instants], columns = ['frame']+header[10:])], ignore_index=True).sort_values('frame') |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1351 tmp = missing.interpolate() |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1352 featureTrajectories = [moving.Trajectory() for i in range(4)] |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1353 for i, r in tmp.iterrows(): |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1354 _, Tr_imu_to_world = oxts[r.frame] |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1355 # transfer_matrix = {'P2': reshape(kittiCalibration['P2'], (3, 4)), |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1356 # 'R_rect': kittiCalibration['R_rect'], |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1357 # 'Tr_cam_to_velo': kittiCalibration['Tr_cam_velo'], |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1358 # # transfer_matrix['Tr_velo_to_cam'] = kittiCalibration['Tr_velo_to_cam'] |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1359 # 'Tr_velo_to_imu': kittiCalibration['Tr_velo_imu'], |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1360 # # transfer_matrix['Tr_imu_to_velo'] = kittiCalibration['Tr_imu_to_velo'] |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1361 # 'Tr_imu_to_world': Tr_imu2w} |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1362 # 3D object |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1363 # compute rotational matrix around yaw axis |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1364 R = roty(r.ry) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1365 |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1366 # 3d bounding box corners |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1367 x_corners = [r.l / 2, r.l / 2, -r.l / 2, -r.l / 2, r.l / 2, r.l / 2, -r.l / 2, -r.l / 2] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1368 y_corners = [0, 0, 0, 0, -r.h, -r.h, -r.h, -r.h] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1369 z_corners = [r.w / 2, -r.w / 2, -r.w / 2, r.w / 2, r.w / 2, -r.w / 2, -r.w / 2, r.w / 2] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1370 # rotate and translate 3d bounding box |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1371 corners3d = dot(R, vstack([x_corners, y_corners, z_corners])) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1372 corners3d[0, :] = corners3d[0, :] + r.x#obj.t[0] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1373 corners3d[1, :] = corners3d[1, :] + r.y#obj.t[1] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1374 corners3d[2, :] = corners3d[2, :] + r.z#obj.t[2] |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1375 # corners3d = transpose(corners3d) |
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1376 # box3d_pts_3d_velo = calib.project_rect_to_velo(box3d_pts_3d) |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1377 refCorners = transpose(dot(invR0, corners3d)) # 8x3 avoid double transpose np.transpose(pts_3d_rect))) in pts_3d_ref = self.project_rect_to_ref(pts_3d_rect) |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1378 homRefCorners = cartesian2Homogeneous(refCorners) |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1379 veloCorners = dot(homRefCorners, transCam2Velo) |
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1380 # self.project_ref_to_velo(pts_3d_ref) |
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1381 # boxes3d_world_single, flag_imu, flag_world = calib_function.project_velo_to_world(bboxes_3d_velo=boxes3d_velo_single, Tr_matrix=transfer_matrix) |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1382 #Tr_velo_to_imu = transfer_matrix['Tr_velo_to_imu'] |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1383 #Tr_imu_to_world = transfer_matrix['Tr_imu_to_world'] |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1384 homVeloCorners = cartesian2Homogeneous(veloCorners) |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1385 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1386 |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1387 homImuCorners = cartesian2Homogeneous(imuCorners) |
1205
3905b393ade0
kitti loading code seems to be working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1204
diff
changeset
|
1388 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1389 |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1390 # take first 4 lines of corners, x,y,_ # x0, y0, _ = boxes3d[0] |
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1391 xCoords = worldCorners[:4,0] |
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1392 yCoords = worldCorners[:4,1] |
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1393 t.addPositionXY(xCoords.mean(), yCoords.mean()) |
1203
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1394 for i in range(4): |
7b3384a8e409
second version of code loading kitti data, to clean
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1202
diff
changeset
|
1395 featureTrajectories[i].addPositionXY(xCoords[i], yCoords[i]) |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1396 # check https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c |
1215
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1397 if interval.length()>1: |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1398 objects.append(moving.MovingObject(num = objNum, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = userType, features = [moving.MovingObject(num = featureNum+i, timeInterval = copy(interval), positions = featureTrajectories[i], velocities = featureTrajectories[i].differentiate(True)) for i in range(4)])) |
1215
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1399 featureNum += 4 |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1400 |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1401 # ego vehicle |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1402 t = moving.Trajectory() |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1403 featureTrajectories = [moving.Trajectory() for i in range(4)] |
1213
3f2214125164
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1212
diff
changeset
|
1404 interval = moving.TimeInterval(0, len(oxts)-1)#int(data.frame.min()), int(data.frame.max())) |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1405 R = roty(pi/2) |
1213
3f2214125164
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1212
diff
changeset
|
1406 #for frame in interval: |
3f2214125164
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1212
diff
changeset
|
1407 for _, Tr_imu_to_world in oxts: |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1408 l = 4.5 # m |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1409 w = 1.8 # m |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1410 h = 0. |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1411 # 3d bounding box corners |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1412 x_corners = [l / 2, l / 2, -l / 2, -l / 2, l / 2, l / 2, -l / 2, -l / 2] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1413 y_corners = [0, 0, 0, 0, -h, -h, -h, -h] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1414 z_corners = [w / 2, -w / 2, -w / 2, w / 2, w / 2, -w / 2, -w / 2, w / 2] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1415 # rotate and translate 3d bounding box |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1416 corners3d = dot(R, vstack([x_corners, y_corners, z_corners])) |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1417 corners3d[0, :] = corners3d[0, :] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1418 corners3d[1, :] = corners3d[1, :] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1419 corners3d[2, :] = corners3d[2, :] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1420 refCorners = transpose(dot(invR0, corners3d)) # 8x3 avoid double transpose np.transpose(pts_3d_rect))) in pts_3d_ref = self.project_rect_to_ref(pts_3d_rect) |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1421 homRefCorners = cartesian2Homogeneous(refCorners) |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1422 veloCorners = dot(homRefCorners, transCam2Velo) |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1423 homVeloCorners = cartesian2Homogeneous(veloCorners) |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1424 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3 |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1425 |
1233
d5695e0b59d9
saving results from ultralytics works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1232
diff
changeset
|
1426 homImuCorners = cartesian2Homogeneous(imuCorners) |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1427 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3 |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1428 |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1429 # take first 4 lines of corners, x,y,_ # x0, y0, _ = boxes3d[0] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1430 xCoords = worldCorners[:4,0] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1431 yCoords = worldCorners[:4,1] |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1432 t.addPositionXY(xCoords.mean(), yCoords.mean()) |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1433 for i in range(4): |
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1434 featureTrajectories[i].addPositionXY(xCoords[i], yCoords[i]) |
1215
1b472cddf9b1
updated reading by interpolating missing frames
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1213
diff
changeset
|
1435 # check https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1436 objects.append(moving.MovingObject(num = max([o.getNum() for o in objects])+1, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = 'Car', features = [moving.MovingObject(num = featureNum+i, timeInterval = copy(interval), positions = featureTrajectories[i], velocities = featureTrajectories[i].differentiate(True)) for i in range(4)])) |
1212
af329f3330ba
work in progress on 3D safety analysis
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1207
diff
changeset
|
1437 |
1202
059b7282aa09
first version of kitti loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1201
diff
changeset
|
1438 return objects |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1439 |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1440 def loadTrajectoriesFromKITTI2D(filename, kittiCalibration = None, oxts = None): |
1251
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1441 '''Loads x,y coordinate series |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1442 e.g. obtained by projecting from image to ground world plane using a homography |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1443 Format: frame_id,track_id,usertype,x,y,score |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1444 |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1445 if oxts is not None, it is obtained using utils.load_oxts_packets_and_poses(['./training/oxts/0001.txt']) from pykitti |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1446 to generate the movement of the ego vehicle''' |
1251
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1447 |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1448 header = ['frame','trackingid','usertype','x','y','score'] |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1449 data = read_csv(filename, delimiter=' ', names = header) |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1450 objects = [] |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1451 for objNum in data.trackingid.unique(): |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1452 tmp = data[data.trackingid == objNum].sort_values('frame') |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1453 interval = moving.TimeInterval(int(tmp.frame.min()), int(tmp.frame.max())) |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1454 userType = tmp.iloc[0].usertype |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1455 if len(tmp) != interval.length(): #interpolate |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1456 print(objNum, len(tmp), interval.length()) |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1457 instants = set(interval).difference(tmp.frame) |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1458 missing = concat([tmp[['frame', 'x', 'y']], DataFrame([[t, NaN, NaN] for t in instants], columns = ['frame', 'x', 'y'])], ignore_index=True).sort_values('frame') |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1459 tmp = missing.interpolate() |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1460 #print(tmp.info()) |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1461 if interval.length()>1: |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1462 t = moving.Trajectory([tmp.x.to_list(), tmp.y.to_list()]) |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1463 objects.append(moving.MovingObject(num = objNum, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = userType)) |
1251
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1464 |
1253
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1465 # ego vehicle |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1466 if kittiCalibration is not None and oxts is not None: |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1467 invR0 = linalg.inv(reshape(kittiCalibration['R_rect'], (3, 3))) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1468 transCam2Velo = transpose(kittiCalibration['Tr_cam_velo']) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1469 t = moving.Trajectory() |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1470 featureTrajectories = [moving.Trajectory() for i in range(4)] |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1471 interval = moving.TimeInterval(0, len(oxts)-1)#int(data.frame.min()), int(data.frame.max())) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1472 #from pykitti.utils import roty |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1473 from trafficintelligence.cvutils import cartesian2Homogeneous |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1474 for _, Tr_imu_to_world in oxts: |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1475 #R = roty(pi/2) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1476 #corners3d = dot(R, [0,0,0]) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1477 #print(corners3d) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1478 #refCorners = transpose(dot(invR0, corners3d)) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1479 #print(invR0, refCorners) # 000 |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1480 #homRefCorners = cartesian2Homogeneous(array([refCorners])) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1481 veloCorners = transCam2Velo[3, :] # dot(homRefCorners, transCam2Velo) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1482 #print(transCam2Velo, veloCorners) # last column of transCam2Velo |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1483 homVeloCorners = cartesian2Homogeneous(veloCorners) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1484 imuCorners = dot(kittiCalibration['Tr_velo_imu'], homVeloCorners.T).T # 8x3 |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1485 #print(kittiCalibration['Tr_velo_imu'], imuCorners) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1486 homImuCorners = cartesian2Homogeneous(imuCorners) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1487 worldCorners = dot(Tr_imu_to_world, homImuCorners.T).T # 8x3 |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1488 #print(Tr_imu_to_world, worldCorners) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1489 xCoords = worldCorners[:4,0] |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1490 yCoords = worldCorners[:4,1] |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1491 t.addPositionXY(xCoords.mean(), yCoords.mean()) |
ef68d4ba7dae
added loading ego vehicle in kitti 2D format and method to plot outline
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1251
diff
changeset
|
1492 objects.append(moving.MovingObject(num = max([o.getNum() for o in objects])+1, timeInterval = interval, positions = t, velocities = t.differentiate(True), userType = 'Car')) |
1251
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1493 return objects |
2b1c8fe8f7e4
added function for KITTI 2D format (just a simple csv-like format with xy coordinates)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1250
diff
changeset
|
1494 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1495 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1496 '''Reads data from the trajectory data provided by NGSIM project |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1497 and converts to our current format.''' |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1498 if append: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1499 out = utils.openCheck(outputfile,'a') |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1500 else: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1501 out = utils.openCheck(outputfile,'w') |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1502 nObjectsPerType = [0,0,0] |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1503 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1504 features = loadNgsimFile(inputfile, sequenceNum) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1505 for f in features: |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1506 nObjectsPerType[f.userType-1] += 1 |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1507 f.write(out) |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1508 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1509 print(nObjectsPerType) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1510 |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1511 out.close() |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1512 |
895
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1513 def loadPinholeCameraModel(filename, tanalystFormat = True): |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1514 '''Loads the data from a file containing the camera parameters |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1515 (pinhole camera model, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1516 and returns a dictionary''' |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1517 if tanalystFormat: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1518 f = utils.openCheck(filename, quitting = True) |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1519 content = utils.getLines(f) |
895
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1520 cameraData = {} |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1521 for l in content: |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1522 tmp = l.split(':') |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1523 cameraData[tmp[0]] = float(tmp[1].strip().replace(',','.')) |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1524 return cameraData |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1525 else: |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1526 print('Unknown camera model (not tanalyst format') |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1527 return None |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1528 |
829 | 1529 def savePositionsToCsv(f, obj): |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1530 timeInterval = obj.getTimeInterval() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1531 positions = obj.getPositions() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1532 curvilinearPositions = obj.getCurvilinearPositions() |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1533 for i in range(int(obj.length())): |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1534 p1 = positions[i] |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1535 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1536 if curvilinearPositions is not None: |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1537 p2 = curvilinearPositions[i] |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1538 s += ',{},{}'.format(p2[0],p2[1]) |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1539 f.write(s+'\n') |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1540 |
829 | 1541 def saveTrajectoriesToCsv(filename, objects): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1542 f = utils.openCheck(filename, 'w') |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1543 for i,obj in enumerate(objects): |
829 | 1544 savePositionsToCsv(f, obj) |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1545 f.close() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1546 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1547 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1548 ######################### |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1549 # Utils to read .ini type text files for configuration, meta data... |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1550 ######################### |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1551 |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1552 class ClassifierParameters(VideoFilenameAddable): |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1553 'Class for the parameters of object classifiers' |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1554 def loadConfigFile(self, filename): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1555 from configparser import ConfigParser |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1556 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1557 config = ConfigParser() |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1558 config.read_file(addSectionHeader(utils.openCheck(filename))) |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1559 |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1560 parentPath = Path(filename).parent |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1561 self.sectionHeader = config.sections()[0] |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1562 |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1563 self.pedBikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'pbv-svm-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1564 self.bikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'bv-svm-filename')) |
1241
ab4c72b9475c
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1239
diff
changeset
|
1565 self.dlFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'dl-filename')) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1566 self.percentIncreaseCrop = config.getfloat(self.sectionHeader, 'percent-increase-crop') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1567 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop') |
1189
ccab20f85710
changed to better parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1182
diff
changeset
|
1568 x = config.getint(self.sectionHeader, 'hog-rescale-size') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1569 self.hogRescaleSize = (x, x) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1570 self.hogNOrientations = config.getint(self.sectionHeader, 'hog-norientations') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1571 x = config.getint(self.sectionHeader, 'hog-npixels-cell') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1572 self.hogNPixelsPerCell = (x, x) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1573 x = config.getint(self.sectionHeader, 'hog-ncells-block') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1574 self.hogNCellsPerBlock = (x, x) |
893
ff92801e5c54
updated hog to scikit-image 0.13 (needed to add a block_norm attribute in classifier.cfg)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
892
diff
changeset
|
1575 self.hogBlockNorm = config.get(self.sectionHeader, 'hog-block-norm') |
1242
4cd8ace3552f
major update for classification, allowing the use of neural network classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1241
diff
changeset
|
1576 self.confidence = config.getfloat(self.sectionHeader, 'confidence') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1577 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1578 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1579 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends') |
1022
b7689372c0ec
renamed quantile to centile
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1021
diff
changeset
|
1580 self.speedAggregationCentile = config.getint(self.sectionHeader, 'speed-aggregation-centile') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1581 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable') |
899
1466a63dd1cf
added a new classification parameter
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
895
diff
changeset
|
1582 self.maxPercentUnknown = config.getfloat(self.sectionHeader, 'max-prop-unknown-appearance') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1583 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1584 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1585 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1586 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1587 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1588 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1589 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1590 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1591 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1592 def __init__(self, filename = None): |
1058
16575ca4537d
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1056
diff
changeset
|
1593 self.configFilename = filename |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1594 if filename is not None and Path(filename).is_file(): |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1595 self.loadConfigFile(filename) |
813
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1596 else: |
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1597 print('Configuration filename {} could not be loaded.'.format(filename)) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1598 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1599 def convertToFrames(self, frameRate, speedRatio = 3.6): |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1600 '''Converts parameters with a relationship to time in 'native' frame time |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1601 speedRatio is the conversion from the speed unit in the config file |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1602 to the distance per second |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1603 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1604 ie param(config file) = speedRatio x fps x param(used in program) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1605 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame''' |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1606 denominator = frameRate*speedRatio |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1607 #denominator2 = denominator**2 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1608 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1609 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1610 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1611 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1612 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1613 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1614 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1615 # special case for the lognormal distribution |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1616 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator) |
854
33d296984dd8
rework and more info on speed probabilities for classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
852
diff
changeset
|
1617 #self.scaleCyclistSpeed = self.scaleCyclistSpeed # no modification of scale |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1618 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1619 |
664
455f9b93819c
added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
660
diff
changeset
|
1620 class ProcessParameters(VideoFilenameAddable): |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1621 '''Class for all parameters controlling data processing: input, |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1622 method parameters, etc. for tracking and safety |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1623 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1624 Note: framerate is already taken into account''' |
1075 | 1625 |
1626 def loadConfigFile(self, filename): | |
1052 | 1627 from configparser import ConfigParser |
1075 | 1628 |
1053
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1629 config = ConfigParser({ 'acceleration-bound' : '3', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1630 'min-velocity-cosine' : '0.8', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1631 'ndisplacements' : '3', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1632 'max-nfeatures' : '1000', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1633 'feature-quality' : '0.0812219538558', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1634 'min-feature-distanceklt' : '3.54964337411', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1635 'block-size' : '7', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1636 'use-harris-detector' : '0', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1637 'k' : '0.04', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1638 'window-size' : '6', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1639 'pyramid-level' : '5', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1640 'min-tracking-error' : '0.183328975142', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1641 'max-number-iterations' : '20', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1642 'feature-flag' : '0', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1643 'min-feature-eig-threshold' : '1e-4', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1644 'min-feature-time' : '15', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1645 'min-feature-displacement' : '0.05', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1646 'tracker-reload-time' : '10'}, strict=False) |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1647 config.read_file(addSectionHeader(utils.openCheck(filename))) |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1648 parentPath = Path(filename).parent |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1649 self.sectionHeader = config.sections()[0] |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1650 # Tracking/display parameters |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1651 self.videoFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'video-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1652 self.databaseFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'database-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1653 self.homographyFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'homography-filename')) |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1654 if Path(self.homographyFilename).is_file(): |
882
4749b71aa7fb
corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
880
diff
changeset
|
1655 self.homography = loadtxt(self.homographyFilename) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1656 else: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1657 self.homography = None |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1658 self.intrinsicCameraFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'intrinsic-camera-filename')) |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1659 if Path(self.intrinsicCameraFilename).is_file(): |
882
4749b71aa7fb
corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
880
diff
changeset
|
1660 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1661 else: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1662 self.intrinsicCameraMatrix = None |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1663 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1664 self.distortionCoefficients = [float(x) for x in distortionCoefficients] |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1665 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication') |
1249
2aa56b101041
added mask functionality for dltrack
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1246
diff
changeset
|
1666 self.maskFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'mask-filename')) |
2aa56b101041
added mask functionality for dltrack
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1246
diff
changeset
|
1667 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1668 self.undistort = config.getboolean(self.sectionHeader, 'undistort') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1669 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1670 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps') |
948
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1671 |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1672 self.classifierFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'classifier-filename')) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1673 |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1674 # Safety parameters |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1675 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate |
976
7f5cbdc107c5
corrected bug in converting time horizon to frames for internal computations (results for TTC, pPET and PET will be in frames)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
948
diff
changeset
|
1676 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1677 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1678 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1679 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1680 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1681 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1682 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1683 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1684 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1685 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1686 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction') |
943
b1e8453c207c
work on motion prediction using motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
938
diff
changeset
|
1687 self.constantSpeedPrototypePrediction = config.getboolean(self.sectionHeader, 'constant-speed') |
948
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1688 self.maxLcssDistance = config.getfloat(self.sectionHeader, 'max-lcss-distance') |
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1689 self.lcssMetric = config.get(self.sectionHeader, 'lcss-metric') |
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1690 self.minLcssSimilarity = config.getfloat(self.sectionHeader, 'min-lcss-similarity') |
1052 | 1691 |
1075 | 1692 # Tracking parameters |
1693 self.accelerationBound = config.getint(self.sectionHeader, 'acceleration-bound') | |
1694 self.minVelocityCosine = config.getfloat(self.sectionHeader, 'min-velocity-cosine') | |
1695 self.ndisplacements = config.getint(self.sectionHeader, 'ndisplacements') | |
1696 self.maxNFeatures = config.getint(self.sectionHeader, 'max-nfeatures') | |
1697 self.minFeatureDistanceKLT = config.getfloat(self.sectionHeader, 'min-feature-distanceklt') | |
1698 self.featureQuality = config.getfloat(self.sectionHeader, 'feature-quality') | |
1699 self.blockSize = config.getint(self.sectionHeader, 'block-size') | |
1700 self.useHarrisDetector = config.getboolean(self.sectionHeader, 'use-harris-detector') | |
1701 self.k = config.getfloat(self.sectionHeader, 'k') | |
1702 self.winSize = config.getint(self.sectionHeader, 'window-size') | |
1703 self.pyramidLevel = config.getint(self.sectionHeader, 'pyramid-level') | |
1704 self.maxNumberTrackingIterations = config.getint(self.sectionHeader, 'max-number-iterations') | |
1705 self.minTrackingError = config.getfloat(self.sectionHeader, 'min-tracking-error') | |
1706 self.featureFlags = config.getboolean(self.sectionHeader, 'feature-flag') | |
1707 self.minFeatureEigThreshold = config.getfloat(self.sectionHeader, 'min-feature-eig-threshold') | |
1708 self.minFeatureTime = config.getint(self.sectionHeader, 'min-feature-time') | |
1709 self.minFeatureDisplacement = config.getfloat(self.sectionHeader, 'min-feature-displacement') | |
1250
77fbd0e2ba7d
dltrack works with moving average filtering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1249
diff
changeset
|
1710 self.smoothingHalfWidth = config.getint(self.sectionHeader, 'smoothing-halfwidth') |
1232
83ca1493d55c
minor work / verif saving works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1216
diff
changeset
|
1711 #self.updateTimer = config.getint(self.sectionHeader, 'tracker-reload-time') |
1075 | 1712 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1713 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1714 def __init__(self, filename = None): |
1058
16575ca4537d
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1056
diff
changeset
|
1715 self.configFilename = filename |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1716 if filename is not None and Path(filename).is_file(): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1717 self.loadConfigFile(filename) |
813
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1718 else: |
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1719 print('Configuration filename {} could not be loaded.'.format(filename)) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1720 |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1721 def processVideoArguments(args): |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1722 '''Loads information from configuration file |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1723 then checks what was passed on the command line |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1724 for override (eg video filename and database filename''' |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1725 if args.configFilename is not None: # consider there is a configuration file |
1177
aa88acf06876
rewrote object concatenation method, cleaner
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1143
diff
changeset
|
1726 parentPath = Path(args.configFilename).parent |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1727 params = ProcessParameters(args.configFilename) |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1728 videoFilename = params.videoFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1729 databaseFilename = params.databaseFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1730 if params.homography is not None: |
1246
2397de73770d
dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1245
diff
changeset
|
1731 homography = params.homography |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1732 invHomography = linalg.inv(params.homography) |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1733 else: |
1246
2397de73770d
dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1245
diff
changeset
|
1734 homography = None |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1735 invHomography = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1736 intrinsicCameraMatrix = params.intrinsicCameraMatrix |
935
0e63a918a1ca
updated classify-objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
927
diff
changeset
|
1737 distortionCoefficients = array(params.distortionCoefficients) |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1738 undistortedImageMultiplication = params.undistortedImageMultiplication |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1739 undistort = params.undistort |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1740 firstFrameNum = params.firstFrameNum |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1741 else: |
1177
aa88acf06876
rewrote object concatenation method, cleaner
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1143
diff
changeset
|
1742 params = None |
1246
2397de73770d
dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1245
diff
changeset
|
1743 homography = None |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1744 invHomography = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1745 undistort = False |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1746 intrinsicCameraMatrix = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1747 distortionCoefficients = [] |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1748 undistortedImageMultiplication = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1749 undistort = False |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1750 firstFrameNum = 0 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1751 |
1245
371c718e57d7
interface updates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1242
diff
changeset
|
1752 # override video and database filenames if present on command line (path is relative to command line location) |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1753 if args.videoFilename is not None: |
1026
73b124160911
more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1022
diff
changeset
|
1754 videoFilename = args.videoFilename |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1755 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1756 videoFilename = params.videoFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1757 if args.databaseFilename is not None: |
1026
73b124160911
more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1022
diff
changeset
|
1758 databaseFilename = args.databaseFilename |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1759 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1760 databaseFilename = params.databaseFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1761 |
1246
2397de73770d
dltrack saves after projecting coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1245
diff
changeset
|
1762 return params, videoFilename, databaseFilename, homography, invHomography, intrinsicCameraMatrix, distortionCoefficients, undistortedImageMultiplication, undistort, firstFrameNum |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1763 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1764 # deprecated |
665
15e244d2a1b5
corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
664
diff
changeset
|
1765 class SceneParameters(object): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1766 def __init__(self, config, sectionName): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1767 from configparser import NoOptionError |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1768 from ast import literal_eval |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1769 try: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1770 self.sitename = config.get(sectionName, 'sitename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1771 self.databaseFilename = config.get(sectionName, 'data-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1772 self.homographyFilename = config.get(sectionName, 'homography-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1773 self.calibrationFilename = config.get(sectionName, 'calibration-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1774 self.videoFilename = config.get(sectionName, 'video-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1775 self.frameRate = config.getfloat(sectionName, 'framerate') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1776 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1777 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0] |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1778 self.rotation = config.getfloat(sectionName, 'rotation') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1779 self.duration = config.getint(sectionName, 'duration') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1780 except NoOptionError as e: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1781 print(e) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1782 print('Not a section for scene meta-data') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1783 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1784 @staticmethod |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1785 def loadConfigFile(filename): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1786 from configparser import ConfigParser |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1787 config = ConfigParser() |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1788 config.readfp(utils.openCheck(filename)) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1789 configDict = dict() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1790 for sectionName in config.sections(): |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1791 configDict[sectionName] = SceneParameters(config, sectionName) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1792 return configDict |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1793 |
382
ba813f148ade
development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
377
diff
changeset
|
1794 |
1201
28bf2420c412
work in progress to import KITTI data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1189
diff
changeset
|
1795 if __name__ == '__main__': |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1796 import doctest |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1797 import unittest |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1798 suite = doctest.DocFileSuite('tests/storage.txt') |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1799 unittest.TextTestRunner().run(suite) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1800 # #doctest.testmod() |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1801 # #doctest.testfile("example.txt") |