Mercurial Hosting > traffic-intelligence
annotate trafficintelligence/storage.py @ 1141:3e0f43edb4d6
Issue loading from different location
author | Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca> |
---|---|
date | Thu, 16 Apr 2020 11:24:35 -0400 |
parents | c4d9c270f999 |
children | 8ac52ebff5f7 |
rev | line source |
---|---|
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1 #! /usr/bin/env python |
208
d9855499fc88
Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents:
204
diff
changeset
|
2 # -*- coding: utf-8 -*- |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
3 '''Various utilities to save and load data''' |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
4 |
1003
75af46516b2b
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
5 from pathlib import Path |
1028
cc5cb04b04b0
major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1026
diff
changeset
|
6 import shutil |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
7 from copy import copy |
417
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
8 import sqlite3, logging |
1141
3e0f43edb4d6
Issue loading from different location
Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca>
parents:
1133
diff
changeset
|
9 from os import chdir |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
10 |
1049
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
11 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt, floor as npfloor, ceil as npceil, linalg, int32, int64 |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
12 from pandas import read_csv, merge |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
13 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
14 from trafficintelligence import utils, moving, events, indicators |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
15 from trafficintelligence.base import VideoFilenameAddable |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
16 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
17 |
50 | 18 ngsimUserTypes = {'twowheels':1, |
204
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
19 'car':2, |
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
20 'truck':3} |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
21 |
908
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
22 tableNames = {'feature':'positions', |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
23 'object': 'objects', |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
24 'objectfeatures': 'positions'} |
b297525b2cbf
added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
901
diff
changeset
|
25 |
1049
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
26 sqlite3.register_adapter(int64, lambda val: int(val)) |
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
27 sqlite3.register_adapter(int32, lambda val: int(val)) |
c9c03c97ed9f
bug fix to store numpy integers in SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1044
diff
changeset
|
28 |
259
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
29 ######################### |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
30 # Sqlite |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
31 ######################### |
8ab76b95ee72
added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
246
diff
changeset
|
32 |
491
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
33 # utils |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
34 def printDBError(error): |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
35 print('DB Error: {}'.format(error)) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
36 |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
37 def dropTables(connection, tableNames): |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
38 'deletes the table with names in tableNames' |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
39 try: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
40 cursor = connection.cursor() |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
41 for tableName in tableNames: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
42 cursor.execute('DROP TABLE IF EXISTS '+tableName) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
43 except sqlite3.OperationalError as error: |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
44 printDBError(error) |
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
45 |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
46 def deleteFromSqlite(filename, dataType): |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
47 'Deletes (drops) some tables in the filename depending on type of data' |
1003
75af46516b2b
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
48 if Path(filename).is_file(): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
49 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
50 if dataType == 'object': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
51 dropTables(connection, ['objects', 'objects_features']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
52 elif dataType == 'interaction': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
53 dropTables(connection, ['interactions', 'indicators']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
54 elif dataType == 'bb': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
55 dropTables(connection, ['bounding_boxes']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
56 elif dataType == 'pois': |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
57 dropTables(connection, ['gaussians2d', 'objects_pois']) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
58 elif dataType == 'prototype': |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
59 dropTables(connection, ['prototypes', 'objects_prototypes', 'features_prototypes']) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
60 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
61 print('Unknown data type {} to delete from database'.format(dataType)) |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
62 else: |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
63 print('{} does not exist'.format(filename)) |
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
64 |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
65 def tableExists(connection, tableName): |
740
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
66 'indicates if the table exists in the database' |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
67 try: |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
68 cursor = connection.cursor() |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
69 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'') |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
70 return cursor.fetchone()[0] == 1 |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
71 except sqlite3.OperationalError as error: |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
72 printDBError(error) |
867bab9f317a
function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
736
diff
changeset
|
73 |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
74 def tableNames(filename): |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
75 'Lists the names of the tables in the SQLite file' |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
76 if Path(filename).is_file(): |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
77 with sqlite3.connect(filename) as connection: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
78 try: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
79 cursor = connection.cursor() |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
80 cursor.execute('SELECT name FROM sqlite_master WHERE type = \'table\'') |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
81 return [row[0] for row in cursor] |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
82 except sqlite3.OperationalError as error: |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
83 printDBError(error) |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
84 return [] |
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
85 |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
86 def createTrajectoryTable(cursor, tableName): |
919
7b3f2e0a2652
saving and loading prototype trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
918
diff
changeset
|
87 if tableName.endswith('positions') or tableName.endswith('velocities'): |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
88 cursor.execute("CREATE TABLE IF NOT EXISTS "+tableName+" (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))") |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
89 else: |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
90 print('Unallowed name {} for trajectory table'.format(tableName)) |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
91 |
879 | 92 def createObjectsTable(cursor): |
93 cursor.execute("CREATE TABLE IF NOT EXISTS objects (object_id INTEGER, road_user_type INTEGER, n_objects INTEGER, PRIMARY KEY(object_id))") | |
94 | |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
95 def createAssignmentTable(cursor, objectType1, objectType2, objectIdColumnName1, objectIdColumnName2): |
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
96 cursor.execute("CREATE TABLE IF NOT EXISTS "+objectType1+"s_"+objectType2+"s ("+objectIdColumnName1+" INTEGER, "+objectIdColumnName2+" INTEGER, PRIMARY KEY("+objectIdColumnName1+","+objectIdColumnName2+"))") |
913
1cd878812529
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
910
diff
changeset
|
97 |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
98 def createObjectsFeaturesTable(cursor): |
913
1cd878812529
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
910
diff
changeset
|
99 cursor.execute("CREATE TABLE IF NOT EXISTS objects_features (object_id INTEGER, trajectory_id INTEGER, PRIMARY KEY(object_id, trajectory_id))") |
879 | 100 |
101 | |
830
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
102 def createCurvilinearTrajectoryTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
103 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))") |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
104 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
105 def createFeatureCorrespondenceTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
106 cursor.execute('CREATE TABLE IF NOT EXISTS feature_correspondences (trajectory_id INTEGER, source_dbname VARCHAR, db_trajectory_id INTEGER, PRIMARY KEY(trajectory_id))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
107 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
108 def createInteractionTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
109 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
110 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
111 def createIndicatorTable(cursor): |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
112 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))') |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
113 |
2a5856961933
first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
829
diff
changeset
|
114 def insertTrajectoryQuery(tableName): |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
115 return "INSERT INTO "+tableName+" VALUES (?,?,?,?)" |
831
a8ff35e6fb43
forgotten change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
830
diff
changeset
|
116 |
879 | 117 def insertObjectQuery(): |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
118 return "INSERT INTO objects VALUES (?,?,?)" |
879 | 119 |
120 def insertObjectFeatureQuery(): | |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
121 return "INSERT INTO objects_features VALUES (?,?)" |
879 | 122 |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
123 def createIndex(connection, tableName, columnName, unique = False): |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
124 '''Creates an index for the column in the table |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
125 I will make querying with a condition on this column faster''' |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
126 try: |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
127 cursor = connection.cursor() |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
128 s = "CREATE " |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
129 if unique: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
130 s += "UNIQUE " |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
131 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")") |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
132 connection.commit() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
133 #connection.close() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
134 except sqlite3.OperationalError as error: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
135 printDBError(error) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
136 |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
137 def getNumberRowsTable(connection, tableName, columnName = None): |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
138 '''Returns the number of rows for the table |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
139 If columnName is not None, means we want the number of distinct values for that column |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
140 (otherwise, we can just count(*))''' |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
141 try: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
142 cursor = connection.cursor() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
143 if columnName is None: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
144 cursor.execute("SELECT COUNT(*) from "+tableName) |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
145 else: |
714
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
146 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
147 return cursor.fetchone()[0] |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
148 except sqlite3.OperationalError as error: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
149 printDBError(error) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
150 |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
151 def getMinMax(connection, tableName, columnName, minmax): |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
152 '''Returns max/min or both for given column in table |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
153 minmax must be string max, min or minmax''' |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
154 try: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
155 cursor = connection.cursor() |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
156 if minmax == 'min' or minmax == 'max': |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
157 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
158 elif minmax == 'minmax': |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
159 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
160 else: |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
161 print("Argument minmax unknown: {}".format(minmax)) |
d6c69d3d09e5
developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
712
diff
changeset
|
162 return cursor.fetchone()[0] |
711
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
163 except sqlite3.OperationalError as error: |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
164 printDBError(error) |
523eda2fafd4
added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
704
diff
changeset
|
165 |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
166 def getObjectCriteria(objectNumbers): |
585
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
167 if objectNumbers is None: |
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
168 query = '' |
aab2242ea88c
minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
584
diff
changeset
|
169 elif type(objectNumbers) == int: |
852
45a53542e046
updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
850
diff
changeset
|
170 query = '<= {0}'.format(objectNumbers-1) |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
171 elif type(objectNumbers) == list: |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
172 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')' |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
173 else: |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
174 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers))) |
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
175 query = '' |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
176 return query |
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
177 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
178 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None, timeStep = None): |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
179 '''Loads trajectories (in the general sense) from the given table |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
180 can be positions or velocities |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
181 |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
182 returns a moving object''' |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
183 cursor = connection.cursor() |
204
966c2cd2bd9f
added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
203
diff
changeset
|
184 |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
185 try: |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
186 objectCriteria = getObjectCriteria(objectNumbers) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
187 queryStatement = None |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
188 if trajectoryType == 'feature': |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
189 queryStatement = 'SELECT * from '+tableName |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
190 if objectNumbers is not None and timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
191 queryStatement += ' WHERE trajectory_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep) |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
192 elif objectNumbers is not None: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
193 queryStatement += ' WHERE trajectory_id '+objectCriteria |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
194 elif timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
195 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep) |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
196 queryStatement += ' ORDER BY trajectory_id, frame_number' |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
197 elif trajectoryType == 'object': |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
198 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
199 if objectNumbers is not None: |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
200 queryStatement += ' AND OF.object_id '+objectCriteria |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
201 if timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
202 queryStatement += ' AND P.frame_number%{} = 0'.format(timeStep) |
712
21aeadcfbabb
added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
711
diff
changeset
|
203 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number' |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
204 elif trajectoryType in ['bbtop', 'bbbottom']: |
587
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
205 if trajectoryType == 'bbtop': |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
206 corner = 'top_left' |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
207 elif trajectoryType == 'bbbottom': |
cf578ba866da
added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
586
diff
changeset
|
208 corner = 'bottom_right' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
209 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
210 if objectNumbers is not None and timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
211 queryStatement += ' WHERE object_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep) |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
212 elif objectNumbers is not None: |
712
21aeadcfbabb
added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
711
diff
changeset
|
213 queryStatement += ' WHERE object_id '+objectCriteria |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
214 elif timeStep is not None: |
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
215 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep) |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
216 queryStatement += ' ORDER BY object_id, frame_number' |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
217 else: |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
218 print('Unknown trajectory type {}'.format(trajectoryType)) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
219 if queryStatement is not None: |
417
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
220 cursor.execute(queryStatement) |
a2ff03a52b73
added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
394
diff
changeset
|
221 logging.debug(queryStatement) |
342
4d69486869a5
work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
341
diff
changeset
|
222 except sqlite3.OperationalError as error: |
491
343cfd185ca6
minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
482
diff
changeset
|
223 printDBError(error) |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
224 return [] |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
225 |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
226 objId = -1 |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
227 obj = None |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
228 objects = [] |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
229 for row in cursor: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
230 if row[0] != objId: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
231 objId = row[0] |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
232 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())): |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
233 objects.append(obj) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
234 elif obj is not None: |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
235 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length())) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
236 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]])) |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
237 else: |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
238 obj.timeInterval.last = row[1] |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
239 obj.positions.addPositionXY(row[2],row[3]) |
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
240 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
241 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())): |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
242 objects.append(obj) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
243 elif obj is not None: |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
244 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length())) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
245 |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
246 return objects |
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
247 |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
248 def loadObjectAttributesFromTable(cursor, objectNumbers, loadNObjects = False): |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
249 objectCriteria = getObjectCriteria(objectNumbers) |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
250 queryStatement = 'SELECT object_id, road_user_type' |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
251 if loadNObjects: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
252 queryStatement += ', n_objects' |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
253 queryStatement += ' FROM objects' |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
254 if objectNumbers is not None: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
255 queryStatement += ' WHERE object_id '+objectCriteria |
693
5ee22bf7e4d5
corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
690
diff
changeset
|
256 cursor.execute(queryStatement) |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
257 attributes = {} |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
258 if loadNObjects: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
259 for row in cursor: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
260 attributes[row[0]] = row[1:] |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
261 else: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
262 for row in cursor: |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
263 attributes[row[0]] = row[1] |
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
264 return attributes |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
265 |
1044
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
266 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False, timeStep = None, nLongestFeaturesPerObject = None): |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
267 '''Loads the trajectories (in the general sense, |
1044
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
268 either features, objects (feature groups), longest features per object, or bounding box series) |
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
269 types are only feature or object |
75a6ad604cc5
work on motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1041
diff
changeset
|
270 if object, features can be loaded with withFeatures or nLongestObjectFeatures used to select the n longest features |
1040
20799ac9524e
integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1039
diff
changeset
|
271 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
272 The number loaded is either the first objectNumbers objects, |
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
273 or the indices in objectNumbers from the database''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
274 objects = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
275 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
276 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
277 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
278 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers, timeStep) |
239
93c26e45efd8
modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
235
diff
changeset
|
279 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
280 if len(objectVelocities) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
281 for o,v in zip(objects, objectVelocities): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
282 if o.getNum() == v.getNum(): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
283 o.velocities = v.positions |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
284 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
285 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
286 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum())) |
263
c71540470057
reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
259
diff
changeset
|
287 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
288 if trajectoryType == 'object': |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
289 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
290 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
291 # attribute feature numbers to objects |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
292 queryStatement = 'SELECT trajectory_id, object_id FROM objects_features' |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
293 if objectNumbers is not None: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
294 queryStatement += ' WHERE object_id '+getObjectCriteria(objectNumbers) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
295 queryStatement += ' ORDER BY object_id' # order is important to group all features per object |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
296 logging.debug(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
297 cursor.execute(queryStatement) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
298 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
299 featureNumbers = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
300 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
301 objId = row[1] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
302 if objId not in featureNumbers: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
303 featureNumbers[objId] = [row[0]] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
304 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
305 featureNumbers[objId].append(row[0]) |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
306 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
307 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
308 obj.featureNumbers = featureNumbers[obj.getNum()] |
685
94b291a5f933
several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
680
diff
changeset
|
309 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
310 # load userType |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
311 attributes = loadObjectAttributesFromTable(cursor, objectNumbers, True) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
312 for obj in objects: |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
313 userType, nObjects = attributes[obj.getNum()] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
314 obj.setUserType(userType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
315 obj.setNObjects(nObjects) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
316 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
317 # add features |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
318 if withFeatures: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
319 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
320 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', obj.featureNumbers, timeStep = timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
321 elif nLongestFeaturesPerObject is not None: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
322 for obj in objects: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
323 queryStatement = 'SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions WHERE trajectory_id '+getObjectCriteria(obj.featureNumbers)+' GROUP BY trajectory_id ORDER BY length DESC' |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
324 logging.debug(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
325 cursor.execute(queryStatement) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
326 obj.features = loadTrajectoriesFromSqlite(filename, 'feature', [row[0] for i,row in enumerate(cursor) if i<nLongestFeaturesPerObject], timeStep = timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
327 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
328 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
329 printDBError(error) |
143
436b87d4b992
wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
78
diff
changeset
|
330 return objects |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
331 |
910
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
332 def loadObjectFeatureFrameNumbers(filename, objectNumbers = None): |
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
333 'Loads the feature frame numbers for each object' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
334 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
335 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
336 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
337 queryStatement = 'SELECT OF.object_id, TL.trajectory_id, TL.length FROM (SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions GROUP BY trajectory_id) TL, objects_features OF WHERE TL.trajectory_id = OF.trajectory_id' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
338 if objectNumbers is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
339 queryStatement += ' AND object_id '+getObjectCriteria(objectNumbers) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
340 queryStatement += ' ORDER BY OF.object_id, TL.length DESC' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
341 logging.debug(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
342 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
343 objectFeatureNumbers = {} |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
344 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
345 objId = row[0] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
346 if objId in objectFeatureNumbers: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
347 objectFeatureNumbers[objId].append(row[1]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
348 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
349 objectFeatureNumbers[objId] = [row[1]] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
350 return objectFeatureNumbers |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
351 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
352 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
353 return None |
910
b58a1061a717
loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
909
diff
changeset
|
354 |
780
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
355 def addCurvilinearTrajectoriesFromSqlite(filename, objects): |
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
356 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane) |
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
357 from a database to an existing MovingObject dict (indexed by each objects's num)''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
358 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
359 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
360 |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
361 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
362 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
363 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
364 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
365 return [] |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
366 |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
367 missingObjectNumbers = [] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
368 objNum = None |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
369 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
370 if objNum != row[0]: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
371 objNum = row[0] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
372 if objNum in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
373 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
374 else: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
375 missingObjectNumbers.append(objNum) |
784
30bd0f2223b7
tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
783
diff
changeset
|
376 if objNum in objects: |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
377 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
378 if len(missingObjectNumbers) > 0: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
379 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers)) |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
380 |
1039
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
381 def saveTrajectoriesToTable(connection, objects, trajectoryType): |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
382 'Saves trajectories in table tableName' |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
383 cursor = connection.cursor() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
384 # Parse feature and/or object structure and commit to DB |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
385 if(trajectoryType == 'feature' or trajectoryType == 'object'): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
386 # Extract features from objects |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
387 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
388 features = [] |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
389 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
390 if obj.hasFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
391 features += obj.getFeatures() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
392 if len(features) == 0: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
393 print('Warning, objects have no features') # todo save centroid trajectories? |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
394 elif trajectoryType == 'feature': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
395 features = objects |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
396 # Setup feature queries |
1039
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
397 createTrajectoryTable(cursor, "positions") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
398 createTrajectoryTable(cursor, "velocities") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
399 positionQuery = insertTrajectoryQuery("positions") |
5621e4ad2428
removing prefix option to loadtrajectories from SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1037
diff
changeset
|
400 velocityQuery = insertTrajectoryQuery("velocities") |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
401 # Setup object queries |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
402 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
403 createObjectsTable(cursor) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
404 createObjectsFeaturesTable(cursor) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
405 objectQuery = insertObjectQuery() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
406 objectFeatureQuery = insertObjectFeatureQuery() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
407 for feature in features: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
408 num = feature.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
409 frameNum = feature.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
410 for p in feature.getPositions(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
411 cursor.execute(positionQuery, (num, frameNum, p.x, p.y)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
412 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
413 velocities = feature.getVelocities() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
414 if velocities is not None: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
415 frameNum = feature.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
416 for v in velocities[:-1]: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
417 cursor.execute(velocityQuery, (num, frameNum, v.x, v.y)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
418 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
419 if trajectoryType == 'object': |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
420 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
421 if obj.hasFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
422 for feature in obj.getFeatures(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
423 featureNum = feature.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
424 cursor.execute(objectFeatureQuery, (obj.getNum(), featureNum)) |
1052 | 425 cursor.execute(objectQuery, (obj.getNum(), obj.getUserType(), obj.nObjects if hasattr(obj, 'nObjects') and obj.nObjects is not None else 1)) |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
426 # Parse curvilinear position structure |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
427 elif(trajectoryType == 'curvilinear'): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
428 createCurvilinearTrajectoryTable(cursor) |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
429 curvilinearQuery = "INSERT INTO curvilinear_positions VALUES (?,?,?,?,?)" |
918
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
430 for obj in objects: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
431 num = obj.getNum() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
432 frameNum = obj.getFirstInstant() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
433 for p in obj.getCurvilinearPositions(): |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
434 cursor.execute(curvilinearQuery, (num, frameNum, p[0], p[1], p[2])) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
435 frameNum += 1 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
436 else: |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
437 print('Unknown trajectory type {}'.format(trajectoryType)) |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
438 connection.commit() |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
439 |
3a06007a4bb7
modularized save trajectories, added slice to Trajectory, etc
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
917
diff
changeset
|
440 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType): |
829 | 441 '''Writes features, ie the trajectory positions (and velocities if exist) |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
442 with their instants to a specified sqlite file |
780
1b22d81ef5ff
cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
778
diff
changeset
|
443 Either feature positions (and velocities if they exist) |
879 | 444 or curvilinear positions will be saved at a time''' |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
445 |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
446 with sqlite3.connect(outputFilename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
447 try: |
1040
20799ac9524e
integrating code of learn-motion-patterns in storage.py
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1039
diff
changeset
|
448 saveTrajectoriesToTable(connection, objects, trajectoryType) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
449 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
450 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
451 |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
452 def setRoadUserTypes(filename, objects): |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
453 '''Saves the user types of the objects in the sqlite database stored in filename |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
454 The objects should exist in the objects table''' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
455 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
456 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
457 for obj in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
458 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum())) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
459 connection.commit() |
778
bd684e57c431
integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
777
diff
changeset
|
460 |
834
119c4efe6398
added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
831
diff
changeset
|
461 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None, timeStep = None): |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
462 '''Loads bounding box moving object from an SQLite |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
463 (format of SQLite output by the ground truth annotation tool |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
464 or Urban Tracker |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
465 |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
466 Load descriptions?''' |
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
467 objects = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
468 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
469 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
470 if objectType == 'bb': |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
471 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
472 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers, timeStep) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
473 userTypes = loadObjectAttributesFromTable(connection.cursor(), objectNumbers) # string format is same as object |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
474 |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
475 for t, b in zip(topCorners, bottomCorners): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
476 num = t.getNum() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
477 if t.getNum() == b.getNum(): |
1133
c4d9c270f999
modification for performance computation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1131
diff
changeset
|
478 annotation = moving.BBMovingObject(num, t, b, t.getTimeInterval(), userTypes[num]) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
479 objects.append(annotation) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
480 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
481 print ('Unknown type of bounding box {}'.format(objectType)) |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
759
diff
changeset
|
482 return objects |
588
c5406edbcf12
added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
587
diff
changeset
|
483 |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
484 def saveInteraction(cursor, interaction): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
485 roadUserNumbers = list(interaction.getRoadUserNumbers()) |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
486 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant())) |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
487 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
488 def saveInteractionsToSqlite(filename, interactions): |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
489 'Saves the interactions in the table' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
490 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
491 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
492 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
493 createInteractionTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
494 for inter in interactions: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
495 saveInteraction(cursor, inter) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
496 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
497 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
498 connection.commit() |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
499 |
341
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
500 def saveIndicator(cursor, interactionNum, indicator): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
501 for instant in indicator.getTimeInterval(): |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
502 if indicator[instant]: |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
503 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant])) |
2f39c4ed0b62
first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
340
diff
changeset
|
504 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
505 def saveIndicatorsToSqlite(filename, interactions, indicatorNames = events.Interaction.indicatorNames): |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
506 'Saves the indicator values in the table' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
507 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
508 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
509 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
510 createInteractionTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
511 createIndicatorTable(cursor) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
512 for inter in interactions: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
513 saveInteraction(cursor, inter) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
514 for indicatorName in indicatorNames: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
515 indicator = inter.getIndicator(indicatorName) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
516 if indicator is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
517 saveIndicator(cursor, inter.getNum(), indicator) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
518 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
519 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
520 connection.commit() |
340
1046b7346886
work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
335
diff
changeset
|
521 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
522 def loadInteractionsFromSqlite(filename): |
482
f6415f012640
adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
417
diff
changeset
|
523 '''Loads interaction and their indicators |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
524 |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
525 TODO choose the interactions to load''' |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
526 interactions = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
527 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
528 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
529 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
530 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
531 cursor.execute('SELECT INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
532 interactionNum = -1 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
533 indicatorTypeNum = -1 |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
534 tmpIndicators = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
535 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
536 if row[0] != interactionNum: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
537 interactionNum = row[0] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
538 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2])) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
539 interactions[-1].indicators = {} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
540 if indicatorTypeNum != row[5] or row[0] != interactionNum: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
541 indicatorTypeNum = row[5] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
542 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
543 indicatorValues = {row[6]:row[7]} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
544 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
545 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
546 indicatorValues[row[6]] = row[7] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
547 interactions[-1].indicators[indicatorName].timeInterval.last = row[6] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
548 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
549 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
550 return [] |
343
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
551 return interactions |
74e437ab5f11
first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
342
diff
changeset
|
552 # load first and last object instants |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
553 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id |
342
4d69486869a5
work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
341
diff
changeset
|
554 |
390
12be4a0cb9aa
sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
382
diff
changeset
|
555 def createBoundingBoxTable(filename, invHomography = None): |
482
f6415f012640
adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
417
diff
changeset
|
556 '''Create the table to store the object bounding boxes in image space |
390
12be4a0cb9aa
sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
382
diff
changeset
|
557 ''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
558 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
559 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
560 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
561 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
562 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from ' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
563 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from ' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
564 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+ |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
565 'GROUP BY object_id, frame_number') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
566 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
567 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
568 connection.commit() |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
569 |
586
ff4f0ce46ca6
modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
585
diff
changeset
|
570 def loadBoundingBoxTableForDisplay(filename): |
647
458890c0437c
cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
646
diff
changeset
|
571 '''Loads bounding boxes from bounding_boxes table for display over trajectories''' |
393
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
572 boundingBoxes = {} # list of bounding boxes for each instant |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
573 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
574 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
575 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
576 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
577 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
578 result = cursor.fetchall() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
579 if len(result) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
580 cursor.execute('SELECT * FROM bounding_boxes') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
581 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
582 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
583 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
584 printDBError(error) |
393
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
585 return boundingBoxes |
eaf7765221d9
added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
390
diff
changeset
|
586 |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
587 ######################### |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
588 # saving and loading for scene interpretation: POIs and Prototypes |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
589 ######################### |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
590 |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
591 def savePrototypesToSqlite(filename, prototypes): |
1033
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
592 '''save the prototypes (a prototype is defined by a filename, a number (id) and type)''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
593 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
594 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
595 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
596 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), nmatchings INTEGER, PRIMARY KEY (prototype_filename, prototype_id, trajectory_type))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
597 for p in prototypes: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
598 cursor.execute('INSERT INTO prototypes VALUES(?,?,?,?)', (p.getFilename(), p.getNum(), p.getTrajectoryType(), p.getNMatchings())) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
599 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
600 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
601 connection.commit() |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
602 |
1033
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
603 def setPrototypeMatchingsInSqlite(filename, prototypes): |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
604 '''updates the prototype matchings''' |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
605 with sqlite3.connect(filename) as connection: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
606 cursor = connection.cursor() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
607 try: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
608 for p in prototypes: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
609 if p.getNMatchings() is None: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
610 nMatchings = 'NULL' |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
611 else: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
612 nMatchings = p.getNMatchings() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
613 cursor.execute('UPDATE prototypes SET nmatchings = {} WHERE prototype_filename = \"{}\" AND prototype_id = {} AND trajectory_type = \"{}\"'.format(nMatchings, p.getFilename(), p.getNum(), p.getTrajectoryType())) |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
614 except sqlite3.OperationalError as error: |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
615 printDBError(error) |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
616 connection.commit() |
8ffb3ae9f3d2
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1030
diff
changeset
|
617 |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
618 def prototypeAssignmentNames(objectType): |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
619 tableName = objectType+'s_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
620 if objectType == 'feature': |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
621 #tableName = 'features_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
622 objectIdColumnName = 'trajectory_id' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
623 elif objectType == 'object': |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
624 #tableName = 'objects_prototypes' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
625 objectIdColumnName = 'object_id' |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
626 return tableName, objectIdColumnName |
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
627 |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
628 def savePrototypeAssignmentsToSqlite(filename, objectNumbers, objectType, labels, prototypes): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
629 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
630 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
631 try: |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
632 tableName, objectIdColumnName = prototypeAssignmentNames(objectType) |
979
cc89267b5ff9
work on learning and assigning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
978
diff
changeset
|
633 cursor.execute('CREATE TABLE IF NOT EXISTS '+tableName+' ('+objectIdColumnName+' INTEGER, prototype_filename VARCHAR, prototype_id INTEGER, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), PRIMARY KEY('+objectIdColumnName+', prototype_filename, prototype_id, trajectory_type))') |
1035
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
634 for objNum, label in zip(objectNumbers, labels): |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
635 if label >=0: |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
636 proto = prototypes[label] |
933588568bec
major update to learn motion pattern, see program description
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1033
diff
changeset
|
637 cursor.execute('INSERT INTO '+tableName+' VALUES(?,?,?,?)', (objNum, proto.getFilename(), proto.getNum(), proto.getTrajectoryType())) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
638 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
639 printDBError(error) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
640 connection.commit() |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
641 |
1037
6a6c37eb3a74
added function to load prototype assignments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1035
diff
changeset
|
642 def loadPrototypeAssignmentsFromSqlite(filename, objectType): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
643 prototypeAssignments = {} |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
644 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
645 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
646 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
647 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
648 tableName, objectIdColumnName = prototypeAssignmentNames(objectType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
649 cursor.execute('SELECT * FROM '+tableName) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
650 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
651 p = moving.Prototype(row[1], row[2], row[3]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
652 if p in prototypeAssignments: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
653 prototypeAssignments[p].append(row[0]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
654 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
655 prototypeAssignments[p] = [row[0]] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
656 return prototypeAssignments |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
657 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
658 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
659 return prototypeAssignments |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
660 |
920
499154254f37
improved prototype loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
919
diff
changeset
|
661 def loadPrototypesFromSqlite(filename, withTrajectories = True): |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
662 'Loads prototype ids and matchings (if stored)' |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
663 prototypes = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
664 if Path(filename).is_file(): |
1141
3e0f43edb4d6
Issue loading from different location
Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca>
parents:
1133
diff
changeset
|
665 filename = str(Path(filename).resolve()) |
3e0f43edb4d6
Issue loading from different location
Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca>
parents:
1133
diff
changeset
|
666 pwd = Path.cwd() |
3e0f43edb4d6
Issue loading from different location
Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca>
parents:
1133
diff
changeset
|
667 chdir(Path(filename).parent) |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
668 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
669 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
670 objects = [] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
671 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
672 cursor.execute('SELECT * FROM prototypes') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
673 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
674 prototypes.append(moving.Prototype(row[0], row[1], row[2], row[3])) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
675 if withTrajectories: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
676 for p in prototypes: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
677 p.setMovingObject(loadTrajectoriesFromSqlite(p.getFilename(), p.getTrajectoryType(), [p.getNum()])[0]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
678 # loadingInformation = {} # complicated slightly optimized |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
679 # for p in prototypes: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
680 # dbfn = p.getFilename() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
681 # trajType = p.getTrajectoryType() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
682 # if (dbfn, trajType) in loadingInformation: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
683 # loadingInformation[(dbfn, trajType)].append(p) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
684 # else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
685 # loadingInformation[(dbfn, trajType)] = [p] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
686 # for k, v in loadingInformation.iteritems(): |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
687 # objects += loadTrajectoriesFromSqlite(k[0], k[1], [p.getNum() for p in v]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
688 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
689 printDBError(error) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
690 if len(set([p.getTrajectoryType() for p in prototypes])) > 1: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
691 print('Different types of prototypes in database ({}).'.format(set([p.getTrajectoryType() for p in prototypes]))) |
1141
3e0f43edb4d6
Issue loading from different location
Etienne Beauchamp <etienne-2.beauchamp@polymtl.ca>
parents:
1133
diff
changeset
|
692 chdir(pwd) |
921
630934595871
work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
920
diff
changeset
|
693 return prototypes |
917
89cc05867c4c
reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
915
diff
changeset
|
694 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
695 def savePOIsToSqlite(filename, gmm, gmmType, gmmId): |
871
6db83beb5350
work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
862
diff
changeset
|
696 '''Saves a Gaussian mixture model (of class sklearn.mixture.GaussianMixture) |
6db83beb5350
work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
862
diff
changeset
|
697 gmmType is a type of GaussianMixture, learnt either from beginnings or ends of trajectories''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
698 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
699 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
700 if gmmType not in ['beginning', 'end']: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
701 print('Unknown POI type {}. Exiting'.format(gmmType)) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
702 import sys |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
703 sys.exit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
704 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
705 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (poi_id INTEGER, id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covariance VARCHAR, covariance_type VARCHAR, weight, precisions_cholesky VARCHAR, PRIMARY KEY(poi_id, id))') |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
706 for i in range(gmm.n_components): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
707 cursor.execute('INSERT INTO gaussians2d VALUES(?,?,?,?,?,?,?,?,?)', (gmmId, i, gmmType, gmm.means_[i][0], gmm.means_[i][1], str(gmm.covariances_[i].tolist()), gmm.covariance_type, gmm.weights_[i], str(gmm.precisions_cholesky_[i].tolist()))) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
708 connection.commit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
709 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
710 printDBError(error) |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
711 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
712 def savePOIAssignmentsToSqlite(filename, objects): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
713 'save the od fields of objects' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
714 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
715 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
716 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
717 cursor.execute('CREATE TABLE IF NOT EXISTS objects_pois (object_id INTEGER, origin_poi_id INTEGER, destination_poi_id INTEGER, PRIMARY KEY(object_id))') |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
718 for o in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
719 cursor.execute('INSERT INTO objects_pois VALUES(?,?,?)', (o.getNum(), o.od[0], o.od[1])) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
720 connection.commit() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
721 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
722 printDBError(error) |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
723 |
927
c030f735c594
added assignment of trajectories to prototypes and cleanup of insert queries
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
922
diff
changeset
|
724 def loadPOIsFromSqlite(filename): |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
725 'Loads all 2D Gaussians in the database' |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
726 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields |
872
c70adaeeddf5
solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
871
diff
changeset
|
727 from ast import literal_eval |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
728 pois = [] |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
729 if Path(filename).is_file(): |
1050
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
730 with sqlite3.connect(filename) as connection: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
731 cursor = connection.cursor() |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
732 try: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
733 cursor.execute('SELECT * from gaussians2d') |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
734 gmmId = None |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
735 gmm = [] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
736 for row in cursor: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
737 if gmmId is None or row[0] != gmmId: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
738 if len(gmm) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
739 tmp = mixture.GaussianMixture(len(gmm), covarianceType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
740 tmp.means_ = array([gaussian['mean'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
741 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
742 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
743 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
744 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
745 pois.append(tmp) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
746 gaussian = {'type': row[2], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
747 'mean': row[3:5], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
748 'covar': array(literal_eval(row[5])), |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
749 'weight': row[7], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
750 'precisions': array(literal_eval(row[8]))} |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
751 gmm = [gaussian] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
752 covarianceType = row[6] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
753 gmmId = row[0] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
754 else: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
755 gmm.append({'type': row[2], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
756 'mean': row[3:5], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
757 'covar': array(literal_eval(row[5])), |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
758 'weight': row[7], |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
759 'precisions': array(literal_eval(row[8]))}) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
760 if len(gmm) > 0: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
761 tmp = mixture.GaussianMixture(len(gmm), covarianceType) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
762 tmp.means_ = array([gaussian['mean'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
763 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
764 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
765 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm] |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
766 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm]) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
767 pois.append(tmp) |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
768 except sqlite3.OperationalError as error: |
9d4a06f49cb8
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1049
diff
changeset
|
769 printDBError(error) |
805
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
770 return pois |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
771 |
180b6b0231c0
added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
784
diff
changeset
|
772 ######################### |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
773 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
774 ######################### |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
775 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
776 def writePrototypesToSqlite(prototypes,nMatching, outputFilename): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
777 ''' prototype dataset is a dictionary with keys== routes, values== prototypes Ids ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
778 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
779 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
780 |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
781 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))') |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
782 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
783 for route in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
784 if prototypes[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
785 for i in prototypes[route]: |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
786 cursor.execute('insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)',(i,route[0],route[1],nMatching[route][i])) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
787 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
788 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
789 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
790 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
791 def readPrototypesFromSqlite(filename): |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
792 ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
793 This function loads the prototype file in the database |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
794 It returns a dictionary for prototypes for each route and nMatching |
915
13434f5017dd
work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
913
diff
changeset
|
795 ''' |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
796 prototypes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
797 nMatching={} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
798 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
799 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
800 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
801 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
802 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
803 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
804 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
805 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
806 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
807 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
808 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
809 route=(row[1],row[2]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
810 if route not in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
811 prototypes[route]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
812 prototypes[route].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
813 nMatching[row[0]]=row[3] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
814 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
815 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
816 return prototypes,nMatching |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
817 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
818 def writeLabelsToSqlite(labels, outputFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
819 """ labels is a dictionary with keys: routes, values: prototypes Ids |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
820 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
821 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
822 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
823 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
824 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
825 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
826 for route in labels: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
827 if labels[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
828 for i in labels[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
829 for j in labels[route][i]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
830 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i)) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
831 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
832 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
833 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
834 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
835 def loadLabelsFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
836 labels = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
837 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
838 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
839 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
840 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
841 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
842 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
843 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
844 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
845 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
846 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
847 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
848 route=(row[1],row[2]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
849 p=row[3] |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
850 if route not in labels: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
851 labels[route]={} |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
852 if p not in labels[route]: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
853 labels[route][p]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
854 labels[route][p].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
855 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
856 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
857 return labels |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
858 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
859 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
860 """ to match the format of second layer prototypes""" |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
861 connection = sqlite3.connect(outFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
862 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
863 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
864 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
865 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
866 for route in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
867 if prototypes[route]!={}: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
868 for i in prototypes[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
869 if prototypes[route][i]!= []: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
870 for j in prototypes[route][i]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
871 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j])) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
872 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
873 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
874 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
875 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
876 def loadSpeedPrototypeFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
877 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
878 This function loads the prototypes table in the database of name <filename>. |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
879 """ |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
880 prototypes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
881 nMatching={} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
882 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
883 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
884 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
885 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
886 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
887 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
888 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
889 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
890 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
891 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
892 route=(row[2],row[3]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
893 if route not in prototypes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
894 prototypes[route]={} |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
895 if row[1] not in prototypes[route]: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
896 prototypes[route][row[1]]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
897 prototypes[route][row[1]].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
898 nMatching[row[0]]=row[4] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
899 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
900 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
901 return prototypes,nMatching |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
902 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
903 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
904 def writeRoutesToSqlite(Routes, outputFilename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
905 """ This function writes the activity path define by start and end IDs""" |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
906 connection = sqlite3.connect(outputFilename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
907 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
908 |
777
ef6dd60be2e1
added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
773
diff
changeset
|
909 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))") |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
910 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
911 for route in Routes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
912 if Routes[route]!=[]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
913 for i in Routes[route]: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
914 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1])) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
915 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
916 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
917 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
918 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
919 def loadRoutesFromSqlite(filename): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
920 Routes = {} |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
921 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
922 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
923 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
924 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
925 try: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
926 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend') |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
927 except sqlite3.OperationalError as error: |
773
bf4a1790cfac
minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
928 printDBError(error) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
929 return [] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
930 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
931 for row in cursor: |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
932 route=(row[1],row[2]) |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
933 if route not in Routes: |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
934 Routes[route]=[] |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
935 Routes[route].append(row[0]) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
936 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
937 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
938 return Routes |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
939 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
940 def setRoutes(filename, objects): |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
941 connection = sqlite3.connect(filename) |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
942 cursor = connection.cursor() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
943 for obj in objects: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
944 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum())) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
945 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum())) |
736
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
946 connection.commit() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
947 connection.close() |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
948 |
967d244968a4
work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
722
diff
changeset
|
949 ######################### |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
950 # txt files |
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
951 ######################### |
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
952 |
1131
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
953 def loadCSVs(filenames, **kwargs): |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
954 '''Loads all the data from the filenames (eg from glob) and returns a concatenated dataframe''' |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
955 data = read_csv(filenames[0], **kwargs) |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
956 for f in filenames[1:]: |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
957 data = data.append(read_csv(filenames[0], **kwargs)) |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
958 return data |
3972d85e3b6c
work on loading data for sumo
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1095
diff
changeset
|
959 |
829 | 960 def saveList(filename, l): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
961 f = utils.openCheck(filename, 'w') |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
962 for x in l: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
963 f.write('{}\n'.format(x)) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
964 f.close() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
965 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
966 def loadListStrings(filename, commentCharacters = utils.commentChar): |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
967 f = utils.openCheck(filename, 'r') |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
968 result = utils.getLines(f, commentCharacters) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
969 f.close() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
970 return result |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
971 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
972 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = utils.commentChar): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
973 values = [] |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
974 for l in loadListStrings(filename, commentCharacters): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
975 if l.startswith(option): |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
976 values.append(l.split(delimiterChar)[1].strip()) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
977 return values |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
978 |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
979 def addSectionHeader(propertiesFile, headerName = 'main'): |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
980 '''Add fake section header |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
981 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
982 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
983 use read_file in Python 3.2+ |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
984 ''' |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
985 yield '[{}]\n'.format(headerName) |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
986 for line in propertiesFile: |
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
987 yield line |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
988 |
892
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
989 def loadPemsTraffic(filename): |
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
990 '''Loads traffic data downloaded from the http://pems.dot.ca.gov clearinghouse |
f766fe0995f4
added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
882
diff
changeset
|
991 into pandas dataframe''' |
1095
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
992 data=read_csv(filename, nrows = 3) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
993 headerNames = ['time', 'station', 'district', 'freeway', 'direction', 'lanetype', 'length', 'nsamples', 'pctobserved', 'totalflow', 'occupancy', 'speed'] # default for 5 min |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
994 nLanes = int((len(data.columns)-len(headerNames))/5) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
995 for i in range(1, nLanes+1): |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
996 headerNames += ['nsamples{}'.format(i), 'flow{}'.format(i), 'occupancy{}'.format(i), 'speed{}'.format(i), 'pctobserved{}'.format(i)] |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
997 return read_csv(filename, names = headerNames) |
e53c6e87bb3f
update pems loader
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1077
diff
changeset
|
998 |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
999 def generatePDLaneColumn(data): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1000 data['LANE'] = data['LANE\\LINK\\NO'].astype(str)+'_'+data['LANE\\INDEX'].astype(str) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1001 |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1002 def convertTrajectoriesVissimToSqlite(filename): |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1003 '''Relies on a system call to sqlite3 |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1004 sqlite3 [file.sqlite] < import_fzp.sql''' |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1005 sqlScriptFilename = "import_fzp.sql" |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1006 # create sql file |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1007 out = utils.openCheck(sqlScriptFilename, "w") |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1008 out.write(".separator \";\"\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1009 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1010 ".import "+filename+" curvilinear_positions\n"+ |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1011 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n") |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1012 out.close() |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1013 # system call |
1004
75601be6019f
work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1003
diff
changeset
|
1014 from subprocess import run |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1015 out = utils.openCheck("err.log", "w") |
1004
75601be6019f
work on process
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1003
diff
changeset
|
1016 run("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out) |
753
3d48e34db846
switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
752
diff
changeset
|
1017 out.close() |
3d48e34db846
switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
752
diff
changeset
|
1018 shutil.os.remove(sqlScriptFilename) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1019 |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1020 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds): |
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1021 '''Finds the ids of the objects that go through any of the link in the list linkIds''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1022 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1023 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1024 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1025 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1026 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1027 return [row[0] for row in cursor] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1028 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1029 printDBError(error) |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1030 |
759
a05b70f307dd
added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
756
diff
changeset
|
1031 def getNObjectsInLinkFromVissimFile(filename, linkIds): |
a05b70f307dd
added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
756
diff
changeset
|
1032 '''Returns the number of objects that traveled through the link ids''' |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1033 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1034 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1035 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1036 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1037 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1038 return {row[0]:row[1] for row in cursor} |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1039 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1040 printDBError(error) |
754
782e8fd3672c
added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
753
diff
changeset
|
1041 |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1042 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True): |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1043 '''Reads data from VISSIM .fzp trajectory file |
755
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1044 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second) |
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1045 for example, there seems to be 10 simulation steps per simulated second in VISSIM, |
f3aeb0b47eff
comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
754
diff
changeset
|
1046 so simulationStepsPerTimeUnit should be 10, |
527
37830a831818
loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
526
diff
changeset
|
1047 so that all times correspond to the number of the simulation step (and can be stored as integers) |
660
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1048 |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1049 Objects positions will be considered only after warmUpLastInstant |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1050 (if the object has no such position, it won't be loaded) |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1051 |
752
14963a9c3b09
debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
751
diff
changeset
|
1052 Assumed to be sorted over time |
14963a9c3b09
debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
751
diff
changeset
|
1053 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id''' |
525
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1054 objects = {} # dictionary of objects index by their id |
7124c7d2a663
first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
524
diff
changeset
|
1055 |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1056 if usePandas: |
718
2cd245cb780d
added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
714
diff
changeset
|
1057 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1058 generatePDLaneColumn(data) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1059 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit |
660
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1060 if warmUpLastInstant is not None: |
994dd644f6ab
corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
649
diff
changeset
|
1061 data = data[data['TIME']>=warmUpLastInstant] |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1062 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False) |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1063 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax}) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1064 for row_index, row in instants.iterrows(): |
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1065 objNum = int(row['NO']) |
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1066 tmp = data[data['NO'] == objNum] |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1067 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']), nObjects = 1) |
642
932f96c89212
added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
1068 # positions should be rounded to nDecimals decimals only |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1069 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist()) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1070 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers: |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1071 return list(objects.values()) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1072 else: |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1073 if filename.endswith(".fzp"): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1074 inputfile = utils.openCheck(filename, quitting = True) |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1075 line = utils.readline(inputfile, '*$') |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1076 while len(line) > 0:#for line in inputfile: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1077 data = line.strip().split(';') |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1078 objNum = int(data[1]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1079 instant = float(data[0])*simulationStepsPerTimeUnit |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1080 s = float(data[4]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1081 y = float(data[5]) |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1082 lane = data[2]+'_'+data[3] |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1083 if objNum not in objects: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1084 if warmUpLastInstant is None or instant >= warmUpLastInstant: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1085 if objectNumbers is None or len(objects) < objectNumbers: |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1086 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1087 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1088 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1089 objects[objNum].timeInterval.last = instant |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1090 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane) |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1091 line = utils.readline(inputfile, '*$') |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1092 elif filename.endswith(".sqlite"): |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1093 with sqlite3.connect(filename) as connection: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1094 cursor = connection.cursor() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1095 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1096 if objectNumbers is not None: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1097 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1098 queryStatement += ' ORDER BY trajectory_id, t' |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1099 try: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1100 cursor.execute(queryStatement) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1101 for row in cursor: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1102 objNum = row[1] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1103 instant = row[0]*simulationStepsPerTimeUnit |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1104 s = row[4] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1105 y = row[5] |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1106 lane = '{}_{}'.format(row[2], row[3]) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1107 if objNum not in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1108 if warmUpLastInstant is None or instant >= warmUpLastInstant: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1109 if objectNumbers is None or len(objects) < objectNumbers: |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1110 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant), nObjects = 1) |
938
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1111 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory() |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1112 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1113 objects[objNum].timeInterval.last = instant |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1114 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane) |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1115 except sqlite3.OperationalError as error: |
fbf12382f3f8
replaced db connection using with
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
935
diff
changeset
|
1116 printDBError(error) |
750
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1117 else: |
6049e9b6902c
work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
744
diff
changeset
|
1118 print("File type of "+filename+" not supported (only .sqlite and .fzp files)") |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1119 return list(objects.values()) |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1120 |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1121 def selectPDLanes(data, lanes = None): |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1122 '''Selects the subset of data for the right lanes |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1123 |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1124 Lane format is a string 'x_y' where x is link index and y is lane index''' |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1125 if lanes is not None: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1126 if 'LANE' not in data.columns: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1127 generatePDLaneColumn(data) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1128 indices = (data['LANE'] == lanes[0]) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1129 for l in lanes[1:]: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1130 indices = indices | (data['LANE'] == l) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1131 return data[indices] |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1132 else: |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1133 return data |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1134 |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1135 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7): |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1136 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1137 and the total number of vehicles |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1138 |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1139 Vehicles are considered finally stationary |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1140 if more than proportionStationaryTime of their total time |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1141 If lanes is not None, only the data for the selected lanes will be provided |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1142 (format as string x_y where x is link index and y is lane index)''' |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1143 if filename.endswith(".fzp"): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1144 columns = ['NO', '$VEHICLE:SIMSEC', 'POS'] |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1145 if lanes is not None: |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1146 columns += ['LANE\\LINK\\NO', 'LANE\\INDEX'] |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1147 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1148 data = selectPDLanes(data, lanes) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1149 data.sort(['$VEHICLE:SIMSEC'], inplace = True) |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1150 |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1151 nStationary = 0 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1152 nVehicles = 0 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1153 for name, group in data.groupby(['NO'], sort = False): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1154 nVehicles += 1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1155 positions = array(group['POS']) |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1156 diff = positions[1:]-positions[:-1] |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1157 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1158 nStationary += 1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1159 elif filename.endswith(".sqlite"): |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1160 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1161 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time? |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1162 pass |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1163 else: |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1164 print("File type of "+filename+" not supported (only .sqlite and .fzp files)") |
646
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1165 |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1166 return nStationary, nVehicles |
6680a69d5bf3
added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
645
diff
changeset
|
1167 |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1168 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True): |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1169 '''Counts the number of collisions per lane in a VISSIM trajectory file |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1170 |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1171 To distinguish between cars passing and collision, |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1172 one checks when the sign of the position difference inverts |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1173 (if the time are closer than collisionTimeDifference) |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1174 If lanes is not None, only the data for the selected lanes will be provided |
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1175 (format as string x_y where x is link index and y is lane index)''' |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1176 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory) |
649
df9ddeaee4a6
added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
647
diff
changeset
|
1177 data = selectPDLanes(data, lanes) |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1178 data = data.convert_objects(convert_numeric=True) |
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1179 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1180 merged = merge(data, data, how='inner', left_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\\LINK\\NO', 'LANE\\INDEX', '$VEHICLE:SIMSEC'], sort = False) |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1181 merged = merged[merged['NO_x']>merged['NO_y']] |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1182 |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1183 nCollisions = 0 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1184 for name, group in merged.groupby(['LANE\\LINK\\NO', 'LANE\\INDEX', 'NO_x', 'NO_y']): |
744
ed6ff2ec0aeb
bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
740
diff
changeset
|
1185 diff = group['POS_x']-group['POS_y'] |
704
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1186 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any |
f83d125d0c55
cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
702
diff
changeset
|
1187 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0: |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1188 xidx = diff[diff < 0].argmax() |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1189 yidx = diff[diff > 0].argmin() |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1190 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference: |
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1191 nCollisions += 1 |
756
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1192 |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1193 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by?? |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1194 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id; |
a73f43aac00e
moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
755
diff
changeset
|
1195 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0 |
645
5ed2118c959d
added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
643
diff
changeset
|
1196 return nCollisions |
524
1dced8932b08
corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
514
diff
changeset
|
1197 |
173
319a04ba9033
function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
143
diff
changeset
|
1198 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1199 '''Reads data from the trajectory data provided by NGSIM project |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1200 and returns the list of Feature objects''' |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1201 objects = [] |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1202 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1203 inputfile = utils.openCheck(filename, quitting = True) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1204 |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1205 def createObject(numbers): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1206 firstFrameNum = int(numbers[1]) |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1207 # do the geometry and usertype |
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1208 |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1209 firstFrameNum = int(numbers[1]) |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1210 lastFrameNum = firstFrameNum+int(numbers[2])-1 |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1211 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1) |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1212 obj = moving.MovingObject(num = int(numbers[0]), |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1213 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum), |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1214 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]), |
1041
fc7c0f38e8a6
added nObjects to MovingObject, with loading/saving
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1040
diff
changeset
|
1215 userType = int(numbers[10]), nObjects = 1) |
78
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1216 obj.userType = int(numbers[10]) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1217 obj.laneNums = [int(numbers[13])] |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1218 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1219 obj.followingVehicles = [int(numbers[15])] # following vehicle (after) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1220 obj.spaceHeadways = [float(numbers[16])] # feet |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1221 obj.timeHeadways = [float(numbers[17])] # seconds |
327
42f2b46ec210
added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
326
diff
changeset
|
1222 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1223 obj.speeds = [float(numbers[11])] |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1224 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1225 return obj |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1226 |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1227 numbers = utils.readline(inputfile).strip().split() |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1228 if (len(numbers) > 0): |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1229 obj = createObject(numbers) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1230 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1231 for line in inputfile: |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1232 numbers = line.strip().split() |
329
a70c205ebdd9
added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
327
diff
changeset
|
1233 if obj.getNum() != int(numbers[0]): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1234 # check and adapt the length to deal with issues in NGSIM data |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1235 if (obj.length() != obj.positions.length()): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1236 print('length pb with object {} ({},{})'.format(obj.getNum(),obj.length(),obj.positions.length())) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1237 obj.last = obj.getFirstInstant()+obj.positions.length()-1 |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1238 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ? |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1239 objects.append(obj) |
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1240 if (nObjects>0) and (len(objects)>=nObjects): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1241 break |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1242 obj = createObject(numbers) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1243 else: |
327
42f2b46ec210
added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
326
diff
changeset
|
1244 obj.laneNums.append(int(numbers[13])) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1245 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7])) |
542
a3add9f751ef
added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
541
diff
changeset
|
1246 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1]) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1247 obj.speeds.append(float(numbers[11])) |
78
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1248 obj.precedingVehicles.append(int(numbers[14])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1249 obj.followingVehicles.append(int(numbers[15])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1250 obj.spaceHeadways.append(float(numbers[16])) |
99e807c29753
added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
72
diff
changeset
|
1251 obj.timeHeadways.append(float(numbers[17])) |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1252 |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1253 if (obj.size[0] != float(numbers[8])): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1254 print('changed length obj {}'.format(obj.getNum())) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1255 if (obj.size[1] != float(numbers[9])): |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1256 print('changed width obj {}'.format(obj.getNum())) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1257 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1258 inputfile.close() |
7
ffddccfab7f9
loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents:
0
diff
changeset
|
1259 return objects |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1260 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1261 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0): |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1262 '''Reads data from the trajectory data provided by NGSIM project |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1263 and converts to our current format.''' |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1264 if append: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1265 out = utils.openCheck(outputfile,'a') |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1266 else: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1267 out = utils.openCheck(outputfile,'w') |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1268 nObjectsPerType = [0,0,0] |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1269 |
564
36605d843be5
modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
552
diff
changeset
|
1270 features = loadNgsimFile(inputfile, sequenceNum) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1271 for f in features: |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1272 nObjectsPerType[f.userType-1] += 1 |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1273 f.write(out) |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1274 |
978
184f1dd307f9
corrected print and exception statements for Python 3
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
976
diff
changeset
|
1275 print(nObjectsPerType) |
0
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1276 |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1277 out.close() |
aed8eb63cdde
initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff
changeset
|
1278 |
895
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1279 def loadPinholeCameraModel(filename, tanalystFormat = True): |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1280 '''Loads the data from a file containing the camera parameters |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1281 (pinhole camera model, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1282 and returns a dictionary''' |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1283 if tanalystFormat: |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1284 f = utils.openCheck(filename, quitting = True) |
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1285 content = utils.getLines(f) |
895
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1286 cameraData = {} |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1287 for l in content: |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1288 tmp = l.split(':') |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1289 cameraData[tmp[0]] = float(tmp[1].strip().replace(',','.')) |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1290 return cameraData |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1291 else: |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1292 print('Unknown camera model (not tanalyst format') |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1293 return None |
739acd338cc0
added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
893
diff
changeset
|
1294 |
829 | 1295 def savePositionsToCsv(f, obj): |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1296 timeInterval = obj.getTimeInterval() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1297 positions = obj.getPositions() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1298 curvilinearPositions = obj.getCurvilinearPositions() |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1299 for i in range(int(obj.length())): |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1300 p1 = positions[i] |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1301 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y) |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
628
diff
changeset
|
1302 if curvilinearPositions is not None: |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1303 p2 = curvilinearPositions[i] |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1304 s += ',{},{}'.format(p2[0],p2[1]) |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1305 f.write(s+'\n') |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1306 |
829 | 1307 def saveTrajectoriesToCsv(filename, objects): |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1308 f = utils.openCheck(filename, 'w') |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1309 for i,obj in enumerate(objects): |
829 | 1310 savePositionsToCsv(f, obj) |
335
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1311 f.close() |
3950bfe22768
added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
330
diff
changeset
|
1312 |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1313 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1314 ######################### |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1315 # Utils to read .ini type text files for configuration, meta data... |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1316 ######################### |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1317 |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1318 class ClassifierParameters(VideoFilenameAddable): |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1319 'Class for the parameters of object classifiers' |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1320 def loadConfigFile(self, filename): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1321 from configparser import ConfigParser |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1322 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1323 config = ConfigParser() |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1324 config.read_file(addSectionHeader(utils.openCheck(filename))) |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1325 |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1326 parentPath = Path(filename).parent |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1327 self.sectionHeader = config.sections()[0] |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1328 |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1329 self.pedBikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'pbv-svm-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1330 self.bikeCarSVMFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'bv-svm-filename')) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1331 self.percentIncreaseCrop = config.getfloat(self.sectionHeader, 'percent-increase-crop') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1332 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1333 x = config.getint(self.sectionHeader, 'hog-rescale-size') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1334 self.hogRescaleSize = (x, x) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1335 self.hogNOrientations = config.getint(self.sectionHeader, 'hog-norientations') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1336 x = config.getint(self.sectionHeader, 'hog-npixels-cell') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1337 self.hogNPixelsPerCell = (x, x) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1338 x = config.getint(self.sectionHeader, 'hog-ncells-block') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1339 self.hogNCellsPerBlock = (x, x) |
893
ff92801e5c54
updated hog to scikit-image 0.13 (needed to add a block_norm attribute in classifier.cfg)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
892
diff
changeset
|
1340 self.hogBlockNorm = config.get(self.sectionHeader, 'hog-block-norm') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1341 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1342 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1343 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends') |
1022
b7689372c0ec
renamed quantile to centile
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1021
diff
changeset
|
1344 self.speedAggregationCentile = config.getint(self.sectionHeader, 'speed-aggregation-centile') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1345 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable') |
899
1466a63dd1cf
added a new classification parameter
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
895
diff
changeset
|
1346 self.maxPercentUnknown = config.getfloat(self.sectionHeader, 'max-prop-unknown-appearance') |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1347 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1348 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1349 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1350 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1351 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1352 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1353 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1354 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed') |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1355 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1356 def __init__(self, filename = None): |
1058
16575ca4537d
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1056
diff
changeset
|
1357 self.configFilename = filename |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1358 if filename is not None and Path(filename).is_file(): |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1359 self.loadConfigFile(filename) |
813
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1360 else: |
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1361 print('Configuration filename {} could not be loaded.'.format(filename)) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1362 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1363 def convertToFrames(self, frameRate, speedRatio = 3.6): |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1364 '''Converts parameters with a relationship to time in 'native' frame time |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1365 speedRatio is the conversion from the speed unit in the config file |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1366 to the distance per second |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1367 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1368 ie param(config file) = speedRatio x fps x param(used in program) |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1369 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame''' |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1370 denominator = frameRate*speedRatio |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1371 #denominator2 = denominator**2 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1372 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1373 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1374 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1375 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1376 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1377 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1378 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1379 # special case for the lognormal distribution |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1380 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator) |
854
33d296984dd8
rework and more info on speed probabilities for classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
852
diff
changeset
|
1381 #self.scaleCyclistSpeed = self.scaleCyclistSpeed # no modification of scale |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1382 |
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1383 |
664
455f9b93819c
added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
660
diff
changeset
|
1384 class ProcessParameters(VideoFilenameAddable): |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1385 '''Class for all parameters controlling data processing: input, |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1386 method parameters, etc. for tracking and safety |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1387 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1388 Note: framerate is already taken into account''' |
1075 | 1389 |
1390 def loadConfigFile(self, filename): | |
1052 | 1391 from configparser import ConfigParser |
1075 | 1392 |
1053
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1393 config = ConfigParser({ 'acceleration-bound' : '3', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1394 'min-velocity-cosine' : '0.8', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1395 'ndisplacements' : '3', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1396 'max-nfeatures' : '1000', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1397 'feature-quality' : '0.0812219538558', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1398 'min-feature-distanceklt' : '3.54964337411', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1399 'block-size' : '7', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1400 'use-harris-detector' : '0', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1401 'k' : '0.04', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1402 'window-size' : '6', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1403 'pyramid-level' : '5', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1404 'min-tracking-error' : '0.183328975142', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1405 'max-number-iterations' : '20', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1406 'feature-flag' : '0', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1407 'min-feature-eig-threshold' : '1e-4', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1408 'min-feature-time' : '15', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1409 'min-feature-displacement' : '0.05', |
60cc87e824c4
Modifying storage.py [configloaders fixing bugs]
Wendlasida
parents:
1052
diff
changeset
|
1410 'tracker-reload-time' : '10'}, strict=False) |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1411 config.read_file(addSectionHeader(utils.openCheck(filename))) |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1412 parentPath = Path(filename).parent |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1413 self.sectionHeader = config.sections()[0] |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1414 # Tracking/display parameters |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1415 self.videoFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'video-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1416 self.databaseFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'database-filename')) |
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1417 self.homographyFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'homography-filename')) |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1418 if Path(self.homographyFilename).is_file(): |
882
4749b71aa7fb
corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
880
diff
changeset
|
1419 self.homography = loadtxt(self.homographyFilename) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1420 else: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1421 self.homography = None |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1422 self.intrinsicCameraFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'intrinsic-camera-filename')) |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1423 if Path(self.intrinsicCameraFilename).is_file(): |
882
4749b71aa7fb
corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
880
diff
changeset
|
1424 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1425 else: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1426 self.intrinsicCameraMatrix = None |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1427 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1428 self.distortionCoefficients = [float(x) for x in distortionCoefficients] |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1429 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1430 self.undistort = config.getboolean(self.sectionHeader, 'undistort') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1431 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1432 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps') |
948
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1433 |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1434 self.classifierFilename = utils.getRelativeFilename(parentPath, config.get(self.sectionHeader, 'classifier-filename')) |
812
21f10332c72b
moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
805
diff
changeset
|
1435 |
537
6c264b914846
work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
536
diff
changeset
|
1436 # Safety parameters |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1437 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate |
976
7f5cbdc107c5
corrected bug in converting time horizon to frames for internal computations (results for TTC, pPET and PET will be in frames)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
948
diff
changeset
|
1438 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1439 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1440 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1441 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1442 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1443 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1444 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1445 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1446 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1447 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1448 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction') |
943
b1e8453c207c
work on motion prediction using motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
938
diff
changeset
|
1449 self.constantSpeedPrototypePrediction = config.getboolean(self.sectionHeader, 'constant-speed') |
948
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1450 self.maxLcssDistance = config.getfloat(self.sectionHeader, 'max-lcss-distance') |
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1451 self.lcssMetric = config.get(self.sectionHeader, 'lcss-metric') |
584b9405e494
added safety analysis parameters for motion patterns
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
943
diff
changeset
|
1452 self.minLcssSimilarity = config.getfloat(self.sectionHeader, 'min-lcss-similarity') |
1052 | 1453 |
1075 | 1454 # Tracking parameters |
1455 self.accelerationBound = config.getint(self.sectionHeader, 'acceleration-bound') | |
1456 self.minVelocityCosine = config.getfloat(self.sectionHeader, 'min-velocity-cosine') | |
1457 self.ndisplacements = config.getint(self.sectionHeader, 'ndisplacements') | |
1458 self.maxNFeatures = config.getint(self.sectionHeader, 'max-nfeatures') | |
1459 self.minFeatureDistanceKLT = config.getfloat(self.sectionHeader, 'min-feature-distanceklt') | |
1460 self.featureQuality = config.getfloat(self.sectionHeader, 'feature-quality') | |
1461 self.blockSize = config.getint(self.sectionHeader, 'block-size') | |
1462 self.useHarrisDetector = config.getboolean(self.sectionHeader, 'use-harris-detector') | |
1463 self.k = config.getfloat(self.sectionHeader, 'k') | |
1464 self.winSize = config.getint(self.sectionHeader, 'window-size') | |
1465 self.pyramidLevel = config.getint(self.sectionHeader, 'pyramid-level') | |
1466 self.maxNumberTrackingIterations = config.getint(self.sectionHeader, 'max-number-iterations') | |
1467 self.minTrackingError = config.getfloat(self.sectionHeader, 'min-tracking-error') | |
1468 self.featureFlags = config.getboolean(self.sectionHeader, 'feature-flag') | |
1469 self.minFeatureEigThreshold = config.getfloat(self.sectionHeader, 'min-feature-eig-threshold') | |
1470 self.minFeatureTime = config.getint(self.sectionHeader, 'min-feature-time') | |
1471 self.minFeatureDisplacement = config.getfloat(self.sectionHeader, 'min-feature-displacement') | |
1472 self.updateTimer = config.getint(self.sectionHeader, 'tracker-reload-time') | |
1473 | |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1474 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1475 def __init__(self, filename = None): |
1058
16575ca4537d
work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1056
diff
changeset
|
1476 self.configFilename = filename |
1065
d4d052a05337
added progress report functionality
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1058
diff
changeset
|
1477 if filename is not None and Path(filename).is_file(): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1478 self.loadConfigFile(filename) |
813
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1479 else: |
ef8795dba5ed
avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
812
diff
changeset
|
1480 print('Configuration filename {} could not be loaded.'.format(filename)) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1481 |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1482 def processVideoArguments(args): |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1483 '''Loads information from configuration file |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1484 then checks what was passed on the command line |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1485 for override (eg video filename and database filename''' |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1486 parentPath = Path(args.configFilename).parent |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1487 if args.configFilename is not None: # consider there is a configuration file |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1488 params = ProcessParameters(args.configFilename) |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1489 videoFilename = params.videoFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1490 databaseFilename = params.databaseFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1491 if params.homography is not None: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1492 invHomography = linalg.inv(params.homography) |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1493 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1494 invHomography = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1495 intrinsicCameraMatrix = params.intrinsicCameraMatrix |
935
0e63a918a1ca
updated classify-objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
927
diff
changeset
|
1496 distortionCoefficients = array(params.distortionCoefficients) |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1497 undistortedImageMultiplication = params.undistortedImageMultiplication |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1498 undistort = params.undistort |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1499 firstFrameNum = params.firstFrameNum |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1500 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1501 invHomography = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1502 undistort = False |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1503 intrinsicCameraMatrix = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1504 distortionCoefficients = [] |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1505 undistortedImageMultiplication = None |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1506 undistort = False |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1507 firstFrameNum = 0 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1508 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1509 # override video and database filenames if present on command line |
1021
16932cefabc1
work on paths in line with new configurations from tracker
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1004
diff
changeset
|
1510 # if not absolute, make all filenames relative to the location of the configuration filename |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1511 if args.videoFilename is not None: |
1026
73b124160911
more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1022
diff
changeset
|
1512 videoFilename = args.videoFilename |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1513 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1514 videoFilename = params.videoFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1515 if args.databaseFilename is not None: |
1026
73b124160911
more plumbing
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1022
diff
changeset
|
1516 databaseFilename = args.databaseFilename |
901
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1517 else: |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1518 databaseFilename = params.databaseFilename |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1519 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1520 return params, videoFilename, databaseFilename, invHomography, intrinsicCameraMatrix, distortionCoefficients, undistortedImageMultiplication, undistort, firstFrameNum |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1521 |
753a081989e2
factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
899
diff
changeset
|
1522 # deprecated |
665
15e244d2a1b5
corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
664
diff
changeset
|
1523 class SceneParameters(object): |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1524 def __init__(self, config, sectionName): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1525 from configparser import NoOptionError |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1526 from ast import literal_eval |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1527 try: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1528 self.sitename = config.get(sectionName, 'sitename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1529 self.databaseFilename = config.get(sectionName, 'data-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1530 self.homographyFilename = config.get(sectionName, 'homography-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1531 self.calibrationFilename = config.get(sectionName, 'calibration-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1532 self.videoFilename = config.get(sectionName, 'video-filename') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1533 self.frameRate = config.getfloat(sectionName, 'framerate') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1534 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1535 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0] |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1536 self.rotation = config.getfloat(sectionName, 'rotation') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1537 self.duration = config.getint(sectionName, 'duration') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1538 except NoOptionError as e: |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1539 print(e) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1540 print('Not a section for scene meta-data') |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1541 |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1542 @staticmethod |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1543 def loadConfigFile(filename): |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
979
diff
changeset
|
1544 from configparser import ConfigParser |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1545 config = ConfigParser() |
1030
aafbc0bab925
moved method around to avoid cross-dependencies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
1546 config.readfp(utils.openCheck(filename)) |
509
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1547 configDict = dict() |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1548 for sectionName in config.sections(): |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1549 configDict[sectionName] = SceneParameters(config, sectionName) |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1550 return configDict |
935430b1d408
corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
491
diff
changeset
|
1551 |
382
ba813f148ade
development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
377
diff
changeset
|
1552 |
215
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1553 if __name__ == "__main__": |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1554 import doctest |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1555 import unittest |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1556 suite = doctest.DocFileSuite('tests/storage.txt') |
5e2983b05d4e
created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
212
diff
changeset
|
1557 unittest.TextTestRunner().run(suite) |
72
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1558 # #doctest.testmod() |
575340e6fce3
corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
50
diff
changeset
|
1559 # #doctest.testfile("example.txt") |