Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 805:180b6b0231c0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added saving/loading points of interests
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Thu, 09 Jun 2016 15:36:21 -0400
parents 30bd0f2223b7
children 21f10332c72b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1 #! /usr/bin/env python
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
2 # -*- coding: utf-8 -*-
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
4
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
5 import utils, moving, events, indicators, shutil
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
6 from base import VideoFilenameAddable
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
7
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
8 import sqlite3, logging
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
9 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
10 from pandas import read_csv, merge
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
11
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
12
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
13 commentChar = '#'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
14
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
15 delimiterChar = '%';
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
16
50
7b06d649122b re-arrangement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 48
diff changeset
17 ngsimUserTypes = {'twowheels':1,
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
18 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
19 'truck':3}
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
20
259
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
21 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
22 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
23 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
24
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
25 # utils
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
26 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
27 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
28
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
29 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
30 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
31 try:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
32 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
33 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
34 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
35 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
36 printDBError(error)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
37
740
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
38 def tableExists(filename, tableName):
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
39 'indicates if the table exists in the database'
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
40 try:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
41 connection = sqlite3.connect(filename)
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
42 cursor = connection.cursor()
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
43 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'')
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
44 return cursor.fetchone()[0] == 1
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
45 except sqlite3.OperationalError as error:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
46 printDBError(error)
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
47
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
48 def createIndex(connection, tableName, columnName, unique = False):
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
49 '''Creates an index for the column in the table
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
50 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
51 try:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
52 #connection = sqlite3.connect(filename)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
53 cursor = connection.cursor()
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
54 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
55 if unique:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
56 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
57 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
58 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
59 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
60 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
61 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
62
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
63 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
64 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
65 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
66 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
67 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
68 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
69 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
70 cursor.execute("SELECT COUNT(*) from "+tableName)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
71 else:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
72 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
73 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
74 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
75 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
76
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
77 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
78 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
79 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
80 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
81 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
82 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
83 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
84 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
85 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
86 else:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
87 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
88 return cursor.fetchone()[0]
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
89 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
90 printDBError(error)
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
91
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
92 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
93 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
94 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
95 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
96 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
97 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
98
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
99 connection = sqlite3.connect(filename)
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
100 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
101
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
102 try:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
103 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
104 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
105 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
106 return []
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
107
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
108 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
109 matched_indexes.append((row[0],row[1]))
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
110
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
111 connection.close()
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
112 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
113
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
114 def getObjectCriteria(objectNumbers):
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
115 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
116 query = ''
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
117 elif type(objectNumbers) == int:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
118 query = 'between 0 and {0}'.format(objectNumbers-1)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
119 elif type(objectNumbers) == list:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
120 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
121 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
122 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
123 query = ''
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
124 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
125
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
126 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None):
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
127 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
128 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
129
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
130 returns a moving object'''
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
131 cursor = connection.cursor()
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
132
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
133 try:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
134 objectCriteria = getObjectCriteria(objectNumbers)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
135 queryStatement = None
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
136 if trajectoryType == 'feature':
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
137 queryStatement = 'SELECT * from '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
138 if objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
139 queryStatement += ' WHERE trajectory_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
140 queryStatement += ' ORDER BY trajectory_id, frame_number'
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
141 elif trajectoryType == 'object':
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
142 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
143 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
144 queryStatement += ' and OF.object_id '+objectCriteria
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
145 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
146 elif trajectoryType in ['bbtop', 'bbbottom']:
587
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
147 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
148 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
149 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
150 corner = 'bottom_right'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
151 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
152 if objectNumbers is not None:
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
153 queryStatement += ' WHERE object_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
154 queryStatement += ' ORDER BY object_id, frame_number'
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
155 else:
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
156 print('Unknown trajectory type {}'.format(trajectoryType))
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
157 if queryStatement is not None:
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
158 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
159 logging.debug(queryStatement)
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
160 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
161 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
162 return []
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
163
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
164 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
165 obj = None
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
166 objects = []
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
167 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
168 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
169 objId = row[0]
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
170 if obj is not None and obj.length() == obj.positions.length():
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
171 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
172 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
173 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
174 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
175 else:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
176 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
177 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
178
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
179 if obj is not None and obj.length() == obj.positions.length():
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
180 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
181 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
182 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
183
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
184 return objects
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
185
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
186 def loadUserTypesFromTable(cursor, trajectoryType, objectNumbers):
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
187 objectCriteria = getObjectCriteria(objectNumbers)
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
188 queryStatement = 'SELECT object_id, road_user_type from objects'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
189 if objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
190 queryStatement += ' WHERE object_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
191 cursor.execute(queryStatement)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
192 userTypes = {}
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
193 for row in cursor:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
194 userTypes[row[0]] = row[1]
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
195 return userTypes
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
196
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
197 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False):
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
198 '''Loads the trajectories (in the general sense,
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
199 either features, objects (feature groups) or bounding box series)
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
200 The number loaded is either the first objectNumbers objects,
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
201 or the indices in objectNumbers from the database'''
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
202 connection = sqlite3.connect(filename)
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
203
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
204 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
205 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
206
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
207 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
208 for o,v in zip(objects, objectVelocities):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
209 if o.getNum() == v.getNum():
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
210 o.velocities = v.positions
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
211 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
212 else:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
213 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
214
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
215 if trajectoryType == 'object':
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
216 cursor = connection.cursor()
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
217 try:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
218 # attribute feature numbers to objects
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
219 objectCriteria = getObjectCriteria(objectNumbers)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
220 queryStatement = 'SELECT P.trajectory_id, OF.object_id from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
221 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
222 queryStatement += ' and OF.object_id '+objectCriteria
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
223 queryStatement += ' group by P.trajectory_id order by OF.object_id' # order is important to group all features per object
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
224 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
225 logging.debug(queryStatement)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
226
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
227 featureNumbers = {}
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
228 for row in cursor:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
229 objId = row[1]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
230 if objId not in featureNumbers:
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
231 featureNumbers[objId] = [row[0]]
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
232 else:
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
233 featureNumbers[objId].append(row[0])
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
234
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
235 for obj in objects:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
236 obj.featureNumbers = featureNumbers[obj.getNum()]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
237
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
238 # load userType
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
239 userTypes = loadUserTypesFromTable(cursor, trajectoryType, objectNumbers)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
240 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
241 obj.userType = userTypes[obj.getNum()]
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
242
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
243 if withFeatures:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
244 nFeatures = 0
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
245 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
246 nFeatures = max(nFeatures, max(obj.featureNumbers))
702
f0a897d7f3a5 corrected bug when loading features with objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 695
diff changeset
247 features = loadTrajectoriesFromSqlite(filename, 'feature', nFeatures+1)
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
248 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
249 obj.setFeatures(features)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
250
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
251 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
252 printDBError(error)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
253 objects = []
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
254
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
255 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
256 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
257
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
258 def addCurvilinearTrajectoriesFromSqlite(filename, objects):
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
259 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane)
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
260 from a database to an existing MovingObject dict (indexed by each objects's num)'''
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
261 connection = sqlite3.connect(filename)
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
262 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
263
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
264 try:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
265 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number')
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
266 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
267 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
268 return []
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
269
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
270 missingObjectNumbers = []
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
271 objNum = None
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
272 for row in cursor:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
273 if objNum != row[0]:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
274 objNum = row[0]
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
275 if objNum in objects:
783
f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 780
diff changeset
276 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 780
diff changeset
277 else:
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
278 missingObjectNumbers.append(objNum)
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
279 if objNum in objects:
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
280 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4])
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
281 if len(missingObjectNumbers) > 0:
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
282 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers))
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
283
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
284 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType, withFeatures = False):
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
285 '''Writes features, ie the trajectories positions (and velocities if exist)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
286 with their instants to a specified sqlite file
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
287 Either feature positions (and velocities if they exist)
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
288 or curvilinear positions will be saved at a time
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
289
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
290 TODO: Not implemented for trajectoryType MovingObject with features
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
291 For objects, with features will control whether the features
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
292 corresponding to the object are also saved'''
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
293
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
294 connection = sqlite3.connect(outputFilename)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
295 try:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
296 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
297
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
298 if trajectoryType == 'feature':
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
299 cursor.execute("CREATE TABLE IF NOT EXISTS positions (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
300 cursor.execute("CREATE TABLE IF NOT EXISTS velocities (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
301
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
302 positionQuery = "insert into positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
303 velocityQuery = "insert into velocities (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
304 for obj in objects:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
305 num = obj.getNum()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
306 frame_number = obj.getFirstInstant()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
307 for position in obj.getPositions():
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
308 cursor.execute(positionQuery, (num, frame_number, position.x, position.y))
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
309 frame_number += 1
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
310 # velocities
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
311 velocities = obj.getVelocities()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
312 if velocities is not None:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
313 frame_number = obj.getFirstInstant()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
314 for i in xrange(velocities.length()-1):
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
315 v = velocities[i]
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
316 cursor.execute(velocityQuery, (num, frame_number, v.x, v.y))
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
317 frame_number += 1
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
318 elif trajectoryType == 'curvilinear':
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
319 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))")
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
320 curvilinearQuery = "insert into curvilinear_positions (trajectory_id, frame_number, s_coordinate, y_coordinate, lane) values (?,?,?,?,?)"
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
321 for obj in objects:
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
322 num = obj.getNum()
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
323 frame_number = obj.getFirstInstant()
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
324 for position in obj.getCurvilinearPositions():
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
325 cursor.execute(curvilinearQuery, (num, frame_number, position[0], position[1], position[2]))
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
326 frame_number += 1
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
327 #elif trajectoryType == 'object':
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
328 else:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
329 print('Unknown trajectory type {}'.format(trajectoryType))
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
330 connection.commit()
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
331 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
332 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
333 connection.close()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
334
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
335 def savePrototypesToSqlite(filename, prototypes, trajectoryType = 'feature'):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
336 'Work in progress, do not use'
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
337 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
338 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
339 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
340 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (id INTEGER PRIMARY KEY, object_id INTEGER, trajectory_id INTEGER, nMatchings INTEGER, FOREIGN KEY(object_id) REFERENCES objects(id), FOREIGN KEY(trajectory_id) REFERENCES positions(trajectory_id))')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
341 #for inter in interactions:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
342 # saveInteraction(cursor, inter)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
343 except sqlite3.OperationalError as error:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
344 printDBError(error)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
345 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
346 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
347
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
348 def loadPrototypesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
349 pass
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
350
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
351 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None):
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
352 '''Loads bounding box moving object from an SQLite
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
353 (format of SQLite output by the ground truth annotation tool
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
354 or Urban Tracker
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
355
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
356 Load descriptions?'''
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
357 connection = sqlite3.connect(filename)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
358 objects = []
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
359
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
360 if objectType == 'bb':
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
361 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
362 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
363 userTypes = loadUserTypesFromTable(connection.cursor(), 'object', objectNumbers) # string format is same as object
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
364
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
365 for t, b in zip(topCorners, bottomCorners):
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
366 num = t.getNum()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
367 if t.getNum() == b.getNum():
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
368 annotation = moving.BBMovingObject(num, t.getTimeInterval(), t, b, userTypes[num])
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
369 objects.append(annotation)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
370 else:
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
371 print ('Unknown type of bounding box {}'.format(objectType))
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
372
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
373 connection.close()
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
374 return objects
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
375
584
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
376 def deleteFromSqlite(filename, dataType):
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
377 'Deletes (drops) some tables in the filename depending on type of data'
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
378 import os
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
379 if os.path.isfile(filename):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
380 connection = sqlite3.connect(filename)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
381 if dataType == 'object':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
382 dropTables(connection, ['objects', 'objects_features'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
383 elif dataType == 'interaction':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
384 dropTables(connection, ['interactions', 'indicators'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
385 elif dataType == 'bb':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
386 dropTables(connection, ['bounding_boxes'])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
387 elif dataType == 'pois':
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
388 dropTables(connection, ['gaussians2d'])
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
389 else:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
390 print('Unknown data type {} to delete from database'.format(dataType))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
391 connection.close()
344
14a2405f54f8 slight modification to safety analysis and generalized script to delete computed data (objects and interactions)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 343
diff changeset
392 else:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
393 print('{} does not exist'.format(filename))
235
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
394
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
395 def createInteractionTable(cursor):
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
396 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
397
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
398 def createIndicatorTables(cursor):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
399 # cursor.execute('CREATE TABLE IF NOT EXISTS indicators (id INTEGER PRIMARY KEY, interaction_id INTEGER, indicator_type INTEGER, FOREIGN KEY(interaction_id) REFERENCES interactions(id))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
400 # cursor.execute('CREATE TABLE IF NOT EXISTS indicator_values (indicator_id INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(indicator_id) REFERENCES indicators(id), PRIMARY KEY(indicator_id, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
401 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
402
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
403 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
404 roadUserNumbers = list(interaction.getRoadUserNumbers())
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
405 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
406
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
407 def saveInteractions(filename, interactions):
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
408 'Saves the interactions in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
409 connection = sqlite3.connect(filename)
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
410 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
411 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
412 createInteractionTable(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
413 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
414 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
415 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
416 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
417 connection.commit()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
418 connection.close()
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
419
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
420 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
421 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
422 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
423 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
424
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
425 def saveIndicators(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
426 'Saves the indicator values in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
427 connection = sqlite3.connect(filename)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
428 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
429 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
430 createInteractionTable(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
431 createIndicatorTables(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
432 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
433 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
434 for indicatorName in indicatorNames:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
435 indicator = inter.getIndicator(indicatorName)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
436 if indicator is not None:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
437 saveIndicator(cursor, inter.getNum(), indicator)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
438 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
439 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
440 connection.commit()
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
441 connection.close()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
442
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
443 def loadInteractions(filename):
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
444 '''Loads interaction and their indicators
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
445
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
446 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
447 interactions = []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
448 connection = sqlite3.connect(filename)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
449 cursor = connection.cursor()
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
450 try:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
451 cursor.execute('select INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
452 interactionNum = -1
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
453 indicatorTypeNum = -1
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
454 tmpIndicators = {}
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
455 for row in cursor:
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
456 if row[0] != interactionNum:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
457 interactionNum = row[0]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
458 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
459 interactions[-1].indicators = {}
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
460 if indicatorTypeNum != row[5] or row[0] != interactionNum:
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
461 indicatorTypeNum = row[5]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
462 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
463 indicatorValues = {row[6]:row[7]}
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
464 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
465 else:
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
466 indicatorValues[row[6]] = row[7]
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
467 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
468 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
469 printDBError(error)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
470 return []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
471 connection.close()
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
472 return interactions
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
473 # load first and last object instants
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
474 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
475
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
476 def createBoundingBoxTable(filename, invHomography = None):
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
477 '''Create the table to store the object bounding boxes in image space
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
478 '''
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
479 connection = sqlite3.connect(filename)
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
480 cursor = connection.cursor()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
481 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
482 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
483 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
484 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from '
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
485 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
486 'GROUP BY object_id, frame_number')
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
487 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
488 printDBError(error)
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
489 connection.commit()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
490 connection.close()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
491
586
ff4f0ce46ca6 modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 585
diff changeset
492 def loadBoundingBoxTableForDisplay(filename):
647
458890c0437c cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 646
diff changeset
493 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
494 connection = sqlite3.connect(filename)
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
495 cursor = connection.cursor()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
496 boundingBoxes = {} # list of bounding boxes for each instant
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
497 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
498 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
499 result = cursor.fetchall()
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
500 if len(result) > 0:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
501 cursor.execute('SELECT * FROM bounding_boxes')
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
502 for row in cursor:
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
503 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
504 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
505 printDBError(error)
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
506 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
507 connection.close()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
508 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
509
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
510 #########################
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
511 # saving and loading for scene interpretation
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
512 #########################
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
513
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
514 def savePOIs(filename, gmm, gmmType, gmmId):
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
515 '''Saves a Gaussian mixture model (of class sklearn.mixture.GMM)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
516 gmmType is a type of GMM, learnt either from beginnings or ends of trajectories'''
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
517 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
518 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
519 if gmmType not in ['beginning', 'end']:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
520 print('Unknown POI type {}. Exiting'.format(gmmType))
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
521 import sys
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
522 sys.exit()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
523 try:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
524 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covar00 REAL, covar01 REAL, covar10 REAL, covar11 REAL, covariance_type VARCHAR, weight, mixture_id INTEGER, PRIMARY KEY(id, mixture_id))')
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
525 for i in xrange(gmm.n_components):
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
526 cursor.execute('INSERT INTO gaussians2d VALUES({}, \'{}\', {}, {}, {}, {}, {}, {}, \'{}\', {}, {})'.format(i, gmmType, gmm.means_[i][0], gmm.means_[i][1], gmm.covars_[i][0,0], gmm.covars_[i][0,1], gmm.covars_[i][1,0], gmm.covars_[i][1,1], gmm.covariance_type, gmm.weights_[i], gmmId))
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
527 connection.commit()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
528 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
529 printDBError(error)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
530 connection.close()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
531
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
532 def loadPOIs(filename):
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
533 'Loads all 2D Gaussians in the database'
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
534 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
535 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
536 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
537 pois = []
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
538 try:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
539 cursor.execute('SELECT * from gaussians2d')
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
540 gmmId = None
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
541 gmm = []
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
542 for row in cursor:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
543 if gmmId is None or row[10] != gmmId:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
544 if len(gmm) > 0:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
545 tmp = mixture.GMM(len(gmm), covarianceType)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
546 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
547 tmp.covars_ = array([gaussian['covar'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
548 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
549 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
550 pois.append(tmp)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
551 gaussian = {'type': row[1],
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
552 'mean': row[2:4],
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
553 'covar': array(row[4:8]).reshape(2,2),
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
554 'weight': row[9]}
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
555 gmm = [gaussian]
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
556 covarianceType = row[8]
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
557 gmmId = row[10]
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
558 else:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
559 gmm.append({'type': row[1],
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
560 'mean': row[2:4],
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
561 'covar': array(row[4:8]).reshape(2,2),
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
562 'weight': row[9]})
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
563 if len(gmm) > 0:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
564 tmp = mixture.GMM(len(gmm), covarianceType)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
565 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
566 tmp.covars_ = array([gaussian['covar'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
567 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
568 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
569 pois.append(tmp)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
570 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
571 printDBError(error)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
572 connection.close()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
573 return pois
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
574
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
575 #########################
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
576 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
577 #########################
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
578
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
579 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
580 """ prototype dataset is a dictionary with keys== routes, values== prototypes Ids """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
581 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
582 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
583
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
584 cursor.execute("CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
585
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
586 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
587 if prototypes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
588 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
589 cursor.execute("insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)",(i,route[0],route[1],nMatching[route][i]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
590
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
591 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
592 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
593
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
594 def readPrototypesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
595 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
596 This function loads the prototype file in the database
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
597 It returns a dictionary for prototypes for each route and nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
598 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
599 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
600 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
601
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
602 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
603 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
604
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
605 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
606 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
607 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
608 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
609 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
610
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
611 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
612 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
613 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
614 prototypes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
615 prototypes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
616 nMatching[row[0]]=row[3]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
617
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
618 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
619 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
620
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
621 def writeLabelsToSqlite(labels, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
622 """ labels is a dictionary with keys: routes, values: prototypes Ids
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
623 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
624 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
625 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
626
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
627 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
628
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
629 for route in labels.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
630 if labels[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
631 for i in labels[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
632 for j in labels[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
633 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
634
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
635 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
636 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
637
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
638 def loadLabelsFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
639 labels = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
640
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
641 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
642 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
643
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
644 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
645 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
646 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
647 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
648 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
649
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
650 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
651 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
652 p=row[3]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
653 if route not in labels.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
654 labels[route]={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
655 if p not in labels[route].keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
656 labels[route][p]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
657 labels[route][p].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
658
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
659 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
660 return labels
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
661
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
662 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
663 """ to match the format of second layer prototypes"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
664 connection = sqlite3.connect(outFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
665 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
666
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
667 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
668
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
669 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
670 if prototypes[route]!={}:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
671 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
672 if prototypes[route][i]!= []:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
673 for j in prototypes[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
674 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
675
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
676 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
677 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
678
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
679 def loadSpeedPrototypeFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
680 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
681 This function loads the prototypes table in the database of name <filename>.
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
682 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
683 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
684 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
685 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
686 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
687
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
688 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
689 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
690 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
691 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
692 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
693
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
694 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
695 route=(row[2],row[3])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
696 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
697 prototypes[route]={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
698 if row[1] not in prototypes[route].keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
699 prototypes[route][row[1]]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
700 prototypes[route][row[1]].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
701 nMatching[row[0]]=row[4]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
702
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
703 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
704 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
705
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
706
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
707 def writeRoutesToSqlite(Routes, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
708 """ This function writes the activity path define by start and end IDs"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
709 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
710 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
711
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
712 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
713
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
714 for route in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
715 if Routes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
716 for i in Routes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
717 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
718
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
719 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
720 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
721
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
722 def loadRoutesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
723 Routes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
724
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
725 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
726 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
727
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
728 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
729 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
730 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
731 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
732 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
733
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
734 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
735 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
736 if route not in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
737 Routes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
738 Routes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
739
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
740 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
741 return Routes
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
742
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
743 def setRoutes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
744 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
745 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
746 for obj in objects:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
747 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum()))
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
748 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum()))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
749 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
750 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
751
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
752 def setRoadUserTypes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
753 '''Saves the user types of the objects in the sqlite database stored in filename
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
754 The objects should exist in the objects table'''
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
755 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
756 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
757 for obj in objects:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
758 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum()))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
759 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
760 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
761
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
762 #########################
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
763 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
764 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
765
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
766 def openCheck(filename, option = 'r', quitting = False):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
767 '''Open file filename in read mode by default
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
768 and checks it is open'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
769 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
770 return open(filename, option)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
771 except IOError:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
772 print 'File %s could not be opened.' % filename
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
773 if quitting:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
774 from sys import exit
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
775 exit()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
776 return None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
777
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
778 def readline(f, commentCharacters = commentChar):
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
779 '''Modified readline function to skip comments
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
780 Can take a list of characters or a string (in will work in both)'''
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
781 s = f.readline()
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
782 while (len(s) > 0) and s[0] in commentCharacters:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
783 s = f.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
784 return s.strip()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
785
722
49e99ca34a7d corrected bugs in old ubc code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 718
diff changeset
786 def getLines(f, delimiterChar = delimiterChar, commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
787 '''Gets a complete entry (all the lines) in between delimiterChar.'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
788 dataStrings = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
789 s = readline(f, commentCharacters)
722
49e99ca34a7d corrected bugs in old ubc code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 718
diff changeset
790 while len(s) > 0 and s[0] != delimiterChar:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
791 dataStrings += [s.strip()]
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
792 s = readline(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
793 return dataStrings
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
794
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
795 def writeList(filename, l):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
796 f = openCheck(filename, 'w')
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
797 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
798 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
799 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
800
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
801 def loadListStrings(filename, commentCharacters = commentChar):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
802 f = openCheck(filename, 'r')
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
803 result = getLines(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
804 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
805 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
806
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
807 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
808 values = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
809 for l in loadListStrings(filename, commentCharacters):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
810 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
811 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
812 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
813
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
814 class FakeSecHead(object):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
815 '''Add fake section header [asection]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
816
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
817 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
818 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
819 '''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
820 def __init__(self, fp):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
821 self.fp = fp
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
822 self.sechead = '[main]\n'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
823
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
824 def readline(self):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
825 if self.sechead:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
826 try: return self.sechead
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
827 finally: self.sechead = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
828 else: return self.fp.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
829
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
830 def generatePDLaneColumn(data):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
831 data['LANE'] = data['LANE\LINK\NO'].astype(str)+'_'+data['LANE\INDEX'].astype(str)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
832
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
833 def convertTrajectoriesVissimToSqlite(filename):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
834 '''Relies on a system call to sqlite3
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
835 sqlite3 [file.sqlite] < import_fzp.sql'''
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
836 sqlScriptFilename = "import_fzp.sql"
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
837 # create sql file
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
838 out = openCheck(sqlScriptFilename, "w")
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
839 out.write(".separator \";\"\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
840 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
841 ".import "+filename+" curvilinear_positions\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
842 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n")
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
843 out.close()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
844 # system call
753
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
845 from subprocess import check_call
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
846 out = openCheck("err.log", "w")
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
847 check_call("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out, shell = True)
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
848 out.close()
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
849 shutil.os.remove(sqlScriptFilename)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
850
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
851 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds):
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
852 '''Finds the ids of the objects that go through any of the link in the list linkIds'''
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
853 connection = sqlite3.connect(filename)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
854 cursor = connection.cursor()
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
855 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')'
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
856 try:
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
857 cursor.execute(queryStatement)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
858 return [row[0] for row in cursor]
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
859 except sqlite3.OperationalError as error:
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
860 printDBError(error)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
861
759
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
862 def getNObjectsInLinkFromVissimFile(filename, linkIds):
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
863 '''Returns the number of objects that traveled through the link ids'''
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
864 connection = sqlite3.connect(filename)
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
865 cursor = connection.cursor()
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
866 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id'
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
867 try:
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
868 cursor.execute(queryStatement)
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
869 return {row[0]:row[1] for row in cursor}
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
870 except sqlite3.OperationalError as error:
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
871 printDBError(error)
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
872
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
873 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
874 '''Reads data from VISSIM .fzp trajectory file
755
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
875 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second)
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
876 for example, there seems to be 10 simulation steps per simulated second in VISSIM,
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
877 so simulationStepsPerTimeUnit should be 10,
527
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
878 so that all times correspond to the number of the simulation step (and can be stored as integers)
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
879
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
880 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
881 (if the object has no such position, it won't be loaded)
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
882
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
883 Assumed to be sorted over time
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
884 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id'''
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
885 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
886
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
887 if usePandas:
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
888 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
889 generatePDLaneColumn(data)
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
890 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
891 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
892 data = data[data['TIME']>=warmUpLastInstant]
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
893 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
894 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
895 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
896 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
897 tmp = data[data['NO'] == objNum]
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
898 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']))
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
899 # positions should be rounded to nDecimals decimals only
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
900 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
901 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers:
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
902 objects.values()
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
903 else:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
904 if filename.endswith(".fzp"):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
905 inputfile = openCheck(filename, quitting = True)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
906 line = readline(inputfile, '*$')
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
907 while len(line) > 0:#for line in inputfile:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
908 data = line.strip().split(';')
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
909 objNum = int(data[1])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
910 instant = float(data[0])*simulationStepsPerTimeUnit
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
911 s = float(data[4])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
912 y = float(data[5])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
913 lane = data[2]+'_'+data[3]
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
914 if objNum not in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
915 if warmUpLastInstant is None or instant >= warmUpLastInstant:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
916 if objectNumbers is None or len(objects) < objectNumbers:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
917 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
918 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
919 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
920 objects[objNum].timeInterval.last = instant
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
921 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
922 line = readline(inputfile, '*$')
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
923 elif filename.endswith(".sqlite"):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
924 connection = sqlite3.connect(filename)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
925 cursor = connection.cursor()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
926 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions'
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
927 if objectNumbers is not None:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
928 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
929 queryStatement += ' ORDER BY trajectory_id, t'
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
930 try:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
931 cursor.execute(queryStatement)
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
932 for row in cursor:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
933 objNum = row[1]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
934 instant = row[0]*simulationStepsPerTimeUnit
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
935 s = row[4]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
936 y = row[5]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
937 lane = '{}_{}'.format(row[2], row[3])
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
938 if objNum not in objects:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
939 if warmUpLastInstant is None or instant >= warmUpLastInstant:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
940 if objectNumbers is None or len(objects) < objectNumbers:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
941 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
942 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
943 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
944 objects[objNum].timeInterval.last = instant
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
945 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
946 except sqlite3.OperationalError as error:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
947 printDBError(error)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
948 else:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
949 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
751
79405a938407 corrected bug
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 750
diff changeset
950 return objects.values()
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
951
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
952 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
953 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
954
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
955 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
956 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
957 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
958 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
959 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
960 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
961 indices = indices | (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
962 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
963 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
964 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
965
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
966 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
967 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
968 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
969
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
970 Vehicles are considered finally stationary
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
971 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
972 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
973 (format as string x_y where x is link index and y is lane index)'''
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
974 if filename.endswith(".fzp"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
975 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
976 if lanes is not None:
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
977 columns += ['LANE\LINK\NO', 'LANE\INDEX']
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
978 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
979 data = selectPDLanes(data, lanes)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
980 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
981
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
982 nStationary = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
983 nVehicles = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
984 for name, group in data.groupby(['NO'], sort = False):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
985 nVehicles += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
986 positions = array(group['POS'])
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
987 diff = positions[1:]-positions[:-1]
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
988 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
989 nStationary += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
990 elif filename.endswith(".sqlite"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
991 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
992 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time?
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
993 pass
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
994 else:
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
995 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
996
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
997 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
998
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
999 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True):
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1000 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1001
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1002 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1003 one checks when the sign of the position difference inverts
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1004 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1005 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1006 (format as string x_y where x is link index and y is lane index)'''
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
1007 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1008 data = selectPDLanes(data, lanes)
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1009 data = data.convert_objects(convert_numeric=True)
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1010
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1011 merged = merge(data, data, how='inner', left_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], sort = False)
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1012 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1013
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1014 nCollisions = 0
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1015 for name, group in merged.groupby(['LANE\LINK\NO', 'LANE\INDEX', 'NO_x', 'NO_y']):
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1016 diff = group['POS_x']-group['POS_y']
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1017 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1018 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1019 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1020 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1021 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1022 nCollisions += 1
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1023
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1024 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by??
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1025 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id;
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1026 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1027 return nCollisions
524
1dced8932b08 corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 514
diff changeset
1028
173
319a04ba9033 function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 143
diff changeset
1029 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1030 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1031 and returns the list of Feature objects'''
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1032 objects = []
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1033
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1034 inputfile = openCheck(filename, quitting = True)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1035
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1036 def createObject(numbers):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1037 firstFrameNum = int(numbers[1])
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1038 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1039
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1040 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1041 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1042 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1043 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1044 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1045 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1046 userType = int(numbers[10]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1047 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1048 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1049 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1050 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1051 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1052 obj.timeHeadways = [float(numbers[17])] # seconds
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1053 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1054 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1055 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1056 return obj
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1057
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1058 numbers = readline(inputfile).strip().split()
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1059 if (len(numbers) > 0):
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1060 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1061
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1062 for line in inputfile:
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1063 numbers = line.strip().split()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1064 if obj.getNum() != int(numbers[0]):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1065 # check and adapt the length to deal with issues in NGSIM data
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1066 if (obj.length() != obj.positions.length()):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1067 print 'length pb with object %s (%d,%d)' % (obj.getNum(),obj.length(),obj.positions.length())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1068 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1069 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1070 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1071 if (nObjects>0) and (len(objects)>=nObjects):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1072 break
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1073 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1074 else:
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1075 obj.laneNums.append(int(numbers[13]))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1076 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542
a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 541
diff changeset
1077 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1078 obj.speeds.append(float(numbers[11]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1079 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1080 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1081 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1082 obj.timeHeadways.append(float(numbers[17]))
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1083
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1084 if (obj.size[0] != float(numbers[8])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1085 print 'changed length obj %d' % (obj.getNum())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1086 if (obj.size[1] != float(numbers[9])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1087 print 'changed width obj %d' % (obj.getNum())
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1088
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1089 inputfile.close()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1090 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1091
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1092 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1093 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1094 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1095 if append:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1096 out = openCheck(outputfile,'a')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1097 else:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1098 out = openCheck(outputfile,'w')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1099 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1100
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1101 features = loadNgsimFile(inputfile, sequenceNum)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1102 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1103 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1104 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1105
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1106 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1107
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1108 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1109
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1110 def writePositionsToCsv(f, obj):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1111 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1112 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1113 curvilinearPositions = obj.getCurvilinearPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1114 for i in xrange(int(obj.length())):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1115 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1116 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1117 if curvilinearPositions is not None:
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1118 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1119 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1120 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1121
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1122 def writeTrajectoriesToCsv(filename, objects):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
1123 f = openCheck(filename, 'w')
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1124 for i,obj in enumerate(objects):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1125 writePositionsToCsv(f, obj)
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1126 f.close()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1127
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1128
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1129 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1130 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1131 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1132
664
455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 660
diff changeset
1133 class ProcessParameters(VideoFilenameAddable):
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1134 '''Class for all parameters controlling data processing: input,
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1135 method parameters, etc. for tracking, classification and safety
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1136
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1137 Note: framerate is already taken into account'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1138
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1139 def loadConfigFile(self, filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1140 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1141 from os import path
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1142
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1143 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1144 config.readfp(FakeSecHead(openCheck(filename)))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1145 self.sectionHeader = config.sections()[0]
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1146 # Tracking/display parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1147 self.videoFilename = config.get(self.sectionHeader, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1148 self.databaseFilename = config.get(self.sectionHeader, 'database-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1149 self.homographyFilename = config.get(self.sectionHeader, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1150 if (path.exists(self.homographyFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1151 self.homography = loadtxt(self.homographyFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1152 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1153 self.homography = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1154 self.intrinsicCameraFilename = config.get(self.sectionHeader, 'intrinsic-camera-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1155 if (path.exists(self.intrinsicCameraFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1156 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1157 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1158 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1159 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1160 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1161 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1162 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1163 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1164 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
377
2aed569f39e7 added utils
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 344
diff changeset
1165
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1166 # Classification parameters
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1167 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1168 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1169 self.speedAggregationQuantile = config.getint(self.sectionHeader, 'speed-aggregation-quantile')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1170 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
1171 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1172 self.pedBikeCarSVMFilename = config.get(self.sectionHeader, 'pbv-svm-filename')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1173 self.bikeCarSVMFilename = config.get(self.sectionHeader, 'bv-svm-filename')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1174 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1175 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1176 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1177 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1178 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1179 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1180 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1181 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1182
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1183 # Safety parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1184 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1185 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1186 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1187 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1188 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1189 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1190 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1191 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1192 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1193 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1194 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1195 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1196
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1197 def __init__(self, filename = None):
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1198 if filename is not None:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1199 self.loadConfigFile(filename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1200
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1201 def convertToFrames(self, speedRatio = 3.6):
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1202 '''Converts parameters with a relationship to time in 'native' frame time
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1203 speedRatio is the conversion from the speed unit in the config file
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1204 to the distance per second
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1205
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1206 ie param(config file) = speedRatio x fps x param(used in program)
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1207 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1208 denominator = self.videoFrameRate*speedRatio
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1209 denominator2 = denominator**2
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1210 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1211 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1212 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1213 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1214 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1215 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1216 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1217 # special case for the lognormal distribution
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1218 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1219 #self.scaleCyclistSpeed = self.scaleCyclistSpeed
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1220
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
1221 class SceneParameters(object):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1222 def __init__(self, config, sectionName):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1223 from ConfigParser import NoOptionError
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1224 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1225 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1226 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1227 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1228 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1229 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1230 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1231 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1232 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1233 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1234 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1235 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1236 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1237 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1238 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1239
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1240 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1241 def loadConfigFile(filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1242 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1243 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1244 config.readfp(openCheck(filename))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1245 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1246 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1247 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1248 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1249
382
ba813f148ade development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 377
diff changeset
1250
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1251 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1252 import doctest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1253 import unittest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1254 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1255 unittest.TextTestRunner().run(suite)
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1256 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1257 # #doctest.testfile("example.txt")