Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 718:2cd245cb780d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added option to set low_memory = False for pandas.read_csv
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Thu, 30 Jul 2015 17:43:14 -0400
parents d6c69d3d09e5
children 49e99ca34a7d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1 #! /usr/bin/env python
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
2 # -*- coding: utf-8 -*-
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
4
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
5 import utils, moving, events, indicators
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
6 from base import VideoFilenameAddable
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
7
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
8 import sqlite3, logging
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
9 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
10
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
11
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
12 commentChar = '#'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
13
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
14 delimiterChar = '%';
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
15
50
7b06d649122b re-arrangement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 48
diff changeset
16 ngsimUserTypes = {'twowheels':1,
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
17 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
18 'truck':3}
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
19
259
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
20 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
21 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
22 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
23
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
24 # utils
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
25 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
26 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
27
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
28 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
29 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
30 try:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
31 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
32 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
33 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
34 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
35 printDBError(error)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
36
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
37 def createIndex(connection, tableName, columnName, unique = False):
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
38 '''Creates an index for the column in the table
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
39 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
40 try:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
41 #connection = sqlite3.connect(filename)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
42 cursor = connection.cursor()
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
43 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
44 if unique:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
45 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
46 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
47 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
48 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
49 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
50 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
51
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
52 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
53 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
54 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
55 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
56 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
57 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
58 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
59 cursor.execute("SELECT COUNT(*) from "+tableName)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
60 else:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
61 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
62 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
63 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
64 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
65
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
66 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
67 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
68 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
69 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
70 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
71 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
72 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
73 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
74 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
75 else:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
76 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
77 return cursor.fetchone()[0]
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
78 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
79 printDBError(error)
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
80
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
81 # TODO: add test if database connection is open
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
82 # IO to sqlite
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
83 def writeTrajectoriesToSqlite(objects, outputFilename, trajectoryType, objectNumbers = -1):
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
84 """
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
85 This function writers trajectories to a specified sqlite file
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
86 @param[in] objects -> a list of trajectories
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
87 @param[in] trajectoryType -
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
88 @param[out] outputFilename -> the .sqlite file containting the written objects
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
89 @param[in] objectNumber : number of objects loaded
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
90 """
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
91 connection = sqlite3.connect(outputFilename)
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
92 cursor = connection.cursor()
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
93
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
94 schema = "CREATE TABLE IF NOT EXISTS \"positions\"(trajectory_id INTEGER,frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))"
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
95 cursor.execute(schema)
212
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
96
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
97 trajectory_id = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
98 frame_number = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
99 if trajectoryType == 'feature':
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
100 if type(objectNumbers) == int and objectNumbers == -1:
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
101 for trajectory in objects:
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
102 trajectory_id += 1
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
103 frame_number = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
104 for position in trajectory.getPositions():
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
105 frame_number += 1
212
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
106 query = "insert into positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
107 cursor.execute(query,(trajectory_id,frame_number,position.x,position.y))
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
108
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
109 connection.commit()
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
110 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
111
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
112 def writeFeaturesToSqlite(objects, outputFilename, trajectoryType, objectNumbers = -1):
552
ca6bded754ac corrected more bugs from merging (caught in regression tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 549
diff changeset
113 '''write features trajectories maintain trajectory ID,velocities dataset '''
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
114 connection = sqlite3.connect(outputFilename)
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
115 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
116
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
117 cursor.execute("CREATE TABLE IF NOT EXISTS \"positions\"(trajectory_id INTEGER,frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
118 cursor.execute("CREATE TABLE IF NOT EXISTS \"velocities\"(trajectory_id INTEGER,frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
119
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
120 if trajectoryType == 'feature':
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
121 if type(objectNumbers) == int and objectNumbers == -1:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
122 for trajectory in objects:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
123 trajectory_id = trajectory.num
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
124 frame_number = trajectory.timeInterval.first
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
125 for position,velocity in zip(trajectory.getPositions(),trajectory.getVelocities()):
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
126 cursor.execute("insert into positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)",(trajectory_id,frame_number,position.x,position.y))
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
127 cursor.execute("insert into velocities (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)",(trajectory_id,frame_number,velocity.x,velocity.y))
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
128 frame_number += 1
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
129
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
130 connection.commit()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
131 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
132
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
133 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
134 """ prototype dataset is a dictionary with keys== routes, values== prototypes Ids """
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
135 connection = sqlite3.connect(outputFilename)
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
136 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
137
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
138 cursor.execute("CREATE TABLE IF NOT EXISTS \"prototypes\"(prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))")
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
139
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
140 for route in prototypes.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
141 if prototypes[route]!=[]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
142 for i in prototypes[route]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
143 cursor.execute("insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)",(i,route[0],route[1],nMatching[route][i]))
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
144
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
145 connection.commit()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
146 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
147
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
148 def loadPrototypesFromSqlite(filename):
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
149 """
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
150 This function loads the prototype file in the database
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
151 It returns a dictionary for prototypes for each route and nMatching
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
152 """
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
153 prototypes = {}
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
154 nMatching={}
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
155
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
156 connection = sqlite3.connect(filename)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
157 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
158
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
159 try:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
160 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
161 except sqlite3.OperationalError as error:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
162 utils.printDBError(error)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
163 return []
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
164
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
165 for row in cursor:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
166 route=(row[1],row[2])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
167 if route not in prototypes.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
168 prototypes[route]=[]
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
169 prototypes[route].append(row[0])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
170 nMatching[row[0]]=row[3]
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
171
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
172 connection.close()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
173 return prototypes,nMatching
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
174
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
175 def writeLabelsToSqlite(labels, outputFilename):
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
176 """ labels is a dictionary with keys: routes, values: prototypes Ids
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
177 """
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
178 connection = sqlite3.connect(outputFilename)
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
179 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
180
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
181 cursor.execute("CREATE TABLE IF NOT EXISTS \"labels\"(object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
182
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
183 for route in labels.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
184 if labels[route]!=[]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
185 for i in labels[route]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
186 for j in labels[route][i]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
187 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
188
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
189 connection.commit()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
190 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
191
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
192 def loadLabelsFromSqlite(filename):
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
193 labels = {}
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
194
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
195 connection = sqlite3.connect(filename)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
196 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
197
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
198 try:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
199 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
200 except sqlite3.OperationalError as error:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
201 utils.printDBError(error)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
202 return []
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
203
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
204 for row in cursor:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
205 route=(row[1],row[2])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
206 p=row[3]
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
207 if route not in labels.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
208 labels[route]={}
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
209 if p not in labels[route].keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
210 labels[route][p]=[]
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
211 labels[route][p].append(row[0])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
212
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
213 connection.close()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
214 return labels
607
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
215 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
216 """ to match the format of second layer prototypes"""
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
217 connection = sqlite3.connect(outFilename)
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
218 cursor = connection.cursor()
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
219
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
220 cursor.execute("CREATE TABLE IF NOT EXISTS \"speedprototypes\"(spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
221
607
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
222 for route in prototypes.keys():
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
223 if prototypes[route]!={}:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
224 for i in prototypes[route]:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
225 if prototypes[route][i]!= []:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
226 for j in prototypes[route][i]:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
227 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
228
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
229 connection.commit()
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
230 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
231
607
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
232 def loadSpeedPrototypeFromSqlite(filename):
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
233 """
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
234 This function loads the prototypes table in the database of name <filename>.
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
235 """
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
236 prototypes = {}
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
237 nMatching={}
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
238 connection = sqlite3.connect(filename)
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
239 cursor = connection.cursor()
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
240
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
241 try:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
242 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
243 except sqlite3.OperationalError as error:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
244 utils.printDBError(error)
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
245 return []
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
246
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
247 for row in cursor:
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
248 route=(row[2],row[3])
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
249 if route not in prototypes.keys():
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
250 prototypes[route]={}
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
251 if row[1] not in prototypes[route].keys():
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
252 prototypes[route][row[1]]=[]
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
253 prototypes[route][row[1]].append(row[0])
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
254 nMatching[row[0]]=row[4]
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
255
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
256 connection.close()
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
257 return prototypes,nMatching
84690dfe5560 add some functions for behaviour analysis
MohamedGomaa
parents: 564
diff changeset
258
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
259
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
260 def writeRoutesToSqlite(Routes, outputFilename):
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
261 """ This function writes the activity path define by start and end IDs"""
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
262 connection = sqlite3.connect(outputFilename)
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
263 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
264
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
265 cursor.execute("CREATE TABLE IF NOT EXISTS \"routes\"(object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
266
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
267 for route in Routes.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
268 if Routes[route]!=[]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
269 for i in Routes[route]:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
270 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
271
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
272 connection.commit()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
273 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
274
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
275 def loadRoutesFromSqlite(filename):
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
276 Routes = {}
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
277
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
278 connection = sqlite3.connect(filename)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
279 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
280
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
281 try:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
282 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
283 except sqlite3.OperationalError as error:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
284 utils.printDBError(error)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
285 return []
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
286
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
287 for row in cursor:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
288 route=(row[1],row[2])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
289 if route not in Routes.keys():
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
290 Routes[route]=[]
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
291 Routes[route].append(row[0])
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
292
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
293 connection.close()
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
294 return Routes
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
295
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
296 def setRoutes(filename, objects):
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
297 connection = sqlite3.connect(filename)
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
298 cursor = connection.cursor()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
299 for obj in objects:
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
300 cursor.execute('update objects set startRouteID = {} where object_id = {}'.format(obj.startRouteID, obj.getNum()))
619
dc2d0a0d7fe1 merged code from Mohamed Gomaa Mohamed for the use of points of interests in mation pattern learning and motion prediction (TRB 2015)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 589 615
diff changeset
301 cursor.execute('update objects set endRouteID = {} where object_id = {}'.format(obj.endRouteID, obj.getNum()))
546
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
302 connection.commit()
6c0923f1ce68 add some functions for behaviour analysis
MohamedGomaa
parents: 537
diff changeset
303 connection.close()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
304
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
305 def setRoadUserTypes(filename, objects):
330
00800ebae698 corrected bug in db loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 329
diff changeset
306 '''Saves the user types of the objects in the sqlite database stored in filename
00800ebae698 corrected bug in db loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 329
diff changeset
307 The objects should exist in the objects table'''
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
308 connection = sqlite3.connect(filename)
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
309 cursor = connection.cursor()
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
310 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
311 cursor.execute('update objects set road_user_type = {} where object_id = {}'.format(obj.getUserType(), obj.getNum()))
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
312 connection.commit()
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
313 connection.close()
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
314
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
315 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
316 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
317 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
318 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
319 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
320 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
321
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
322 connection = sqlite3.connect(filename)
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
323 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
324
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
325 try:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
326 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
327 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
328 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
329 return []
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
330
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
331 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
332 matched_indexes.append((row[0],row[1]))
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
333
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
334 connection.close()
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
335 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
336
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
337 def getObjectCriteria(objectNumbers):
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
338 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
339 query = ''
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
340 elif type(objectNumbers) == int:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
341 query = 'between 0 and {0}'.format(objectNumbers-1)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
342 elif type(objectNumbers) == list:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
343 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
344 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
345 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
346 query = ''
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
347 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
348
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
349 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None):
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
350 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
351 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
352
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
353 returns a moving object'''
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
354 cursor = connection.cursor()
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
355
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
356 try:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
357 objectCriteria = getObjectCriteria(objectNumbers)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
358 if trajectoryType == 'feature':
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
359 queryStatement = 'SELECT * from '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
360 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
361 queryStatement += ' where trajectory_id '+objectCriteria
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
362 queryStatement += ' ORDER BY trajectory_id, frame_number'
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
363 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
364 logging.debug(queryStatement)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
365 elif trajectoryType == 'object':
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
366 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF where P.trajectory_id = OF.trajectory_id'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
367 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
368 queryStatement += ' and OF.object_id '+objectCriteria
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
369 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
587
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
370 cursor.execute(queryStatement)
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
371 logging.debug(queryStatement)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
372 elif trajectoryType in ['bbtop', 'bbbottom']:
587
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
373 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
374 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
375 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
376 corner = 'bottom_right'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
377 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
378 if objectNumbers is not None:
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
379 queryStatement += ' WHERE object_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
380 queryStatement += ' ORDER BY object_id, frame_number'
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
381 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
382 logging.debug(queryStatement)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
383 else:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
384 print('no trajectory type was chosen')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
385 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
386 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
387 return []
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
388
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
389 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
390 obj = None
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
391 objects = []
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
392 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
393 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
394 objId = row[0]
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
395 if obj is not None and obj.length() == obj.positions.length():
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
396 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
397 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
398 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
399 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
400 else:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
401 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
402 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
403
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
404 if obj is not None and obj.length() == obj.positions.length():
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
405 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
406 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
407 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
408
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
409 return objects
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
410
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
411 def loadUserTypesFromTable(cursor, trajectoryType, objectNumbers):
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
412 objectCriteria = getObjectCriteria(objectNumbers)
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
413 queryStatement = 'SELECT object_id, road_user_type from objects'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
414 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
415 queryStatement += ' where object_id '+objectCriteria
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
416 cursor.execute(queryStatement)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
417 userTypes = {}
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
418 for row in cursor:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
419 userTypes[row[0]] = row[1]
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
420 return userTypes
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
421
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
422 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False):
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
423 '''Loads the first objectNumbers objects or the indices in objectNumbers from the database'''
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
424 connection = sqlite3.connect(filename)
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
425
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
426 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
427 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
428
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
429 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
430 for o,v in zip(objects, objectVelocities):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
431 if o.getNum() == v.getNum():
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
432 o.velocities = v.positions
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
433 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
434 else:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
435 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
436
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
437 if trajectoryType == 'object':
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
438 cursor = connection.cursor()
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
439 try:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
440 # attribute feature numbers to objects
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
441 objectCriteria = getObjectCriteria(objectNumbers)
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
442 queryStatement = 'SELECT P.trajectory_id, OF.object_id from positions P, objects_features OF where P.trajectory_id = OF.trajectory_id'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
443 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
444 queryStatement += ' and OF.object_id '+objectCriteria
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
445 queryStatement += ' group by P.trajectory_id order by OF.object_id' # order is important to group all features per object
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
446 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
447 logging.debug(queryStatement)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
448
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
449 featureNumbers = {}
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
450 for row in cursor:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
451 objId = row[1]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
452 if objId not in featureNumbers:
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
453 featureNumbers[objId] = [row[0]]
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
454 else:
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
455 featureNumbers[objId].append(row[0])
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
456
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
457 for obj in objects:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
458 obj.featureNumbers = featureNumbers[obj.getNum()]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
459
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
460 # load userType
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
461 userTypes = loadUserTypesFromTable(cursor, trajectoryType, objectNumbers)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
462 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
463 obj.userType = userTypes[obj.getNum()]
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
464
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
465 if withFeatures:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
466 nFeatures = 0
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
467 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
468 nFeatures = max(nFeatures, max(obj.featureNumbers))
702
f0a897d7f3a5 corrected bug when loading features with objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 695
diff changeset
469 features = loadTrajectoriesFromSqlite(filename, 'feature', nFeatures+1)
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
470 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
471 obj.setFeatures(features)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
472
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
473 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
474 printDBError(error)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
475 objects = []
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
476
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
477 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
478 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
479
589
5800a87f11ae corrected one bug and changed attribute names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 588
diff changeset
480 def loadGroundTruthFromSqlite(filename, gtType = 'bb', gtNumbers = None):
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
481 'Loads bounding box annotations (ground truth) from an SQLite '
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
482 connection = sqlite3.connect(filename)
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
483 gt = []
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
484
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
485 if gtType == 'bb':
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
486 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', gtNumbers)
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
487 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', gtNumbers)
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
488 userTypes = loadUserTypesFromTable(connection.cursor(), 'object', gtNumbers) # string format is same as object
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
489
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
490 for t, b in zip(topCorners, bottomCorners):
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
491 num = t.getNum()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
492 if t.getNum() == b.getNum():
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
493 annotation = moving.BBAnnotation(num, t.getTimeInterval(), t, b, userTypes[num])
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
494 gt.append(annotation)
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
495 else:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
496 print ('Unknown type of annotation {}'.format(gtType))
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
497
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
498 connection.close()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
499 return gt
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
500
584
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
501 def deleteFromSqlite(filename, dataType):
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
502 'Deletes (drops) some tables in the filename depending on type of data'
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
503 import os
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
504 if os.path.isfile(filename):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
505 connection = sqlite3.connect(filename)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
506 if dataType == 'object':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
507 dropTables(connection, ['objects', 'objects_features'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
508 elif dataType == 'interaction':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
509 dropTables(connection, ['interactions', 'indicators'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
510 elif dataType == 'bb':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
511 dropTables(connection, ['bounding_boxes'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
512 else:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
513 print('Unknown data type {} to delete from database'.format(dataType))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
514 connection.close()
344
14a2405f54f8 slight modification to safety analysis and generalized script to delete computed data (objects and interactions)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 343
diff changeset
515 else:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
516 print('{} does not exist'.format(filename))
235
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
517
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
518 def createInteractionTable(cursor):
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
519 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
520
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
521 def createIndicatorTables(cursor):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
522 # cursor.execute('CREATE TABLE IF NOT EXISTS indicators (id INTEGER PRIMARY KEY, interaction_id INTEGER, indicator_type INTEGER, FOREIGN KEY(interaction_id) REFERENCES interactions(id))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
523 # cursor.execute('CREATE TABLE IF NOT EXISTS indicator_values (indicator_id INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(indicator_id) REFERENCES indicators(id), PRIMARY KEY(indicator_id, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
524 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
525
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
526 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
527 roadUserNumbers = list(interaction.getRoadUserNumbers())
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
528 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
529
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
530 def saveInteractions(filename, interactions):
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
531 'Saves the interactions in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
532 connection = sqlite3.connect(filename)
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
533 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
534 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
535 createInteractionTable(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
536 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
537 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
538 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
539 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
540 connection.commit()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
541 connection.close()
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
542
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
543 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
544 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
545 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
546 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
547
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
548 def saveIndicators(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
549 'Saves the indicator values in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
550 connection = sqlite3.connect(filename)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
551 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
552 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
553 createInteractionTable(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
554 createIndicatorTables(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
555 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
556 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
557 for indicatorName in indicatorNames:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
558 indicator = inter.getIndicator(indicatorName)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
559 if indicator is not None:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
560 saveIndicator(cursor, inter.getNum(), indicator)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
561 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
562 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
563 connection.commit()
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
564 connection.close()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
565
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
566 def loadInteractions(filename):
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
567 '''Loads interaction and their indicators
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
568
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
569 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
570 interactions = []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
571 connection = sqlite3.connect(filename)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
572 cursor = connection.cursor()
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
573 try:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
574 cursor.execute('select INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND where INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
575 interactionNum = -1
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
576 indicatorTypeNum = -1
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
577 tmpIndicators = {}
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
578 for row in cursor:
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
579 if row[0] != interactionNum:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
580 interactionNum = row[0]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
581 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
582 interactions[-1].indicators = {}
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
583 if indicatorTypeNum != row[5] or row[0] != interactionNum:
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
584 indicatorTypeNum = row[5]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
585 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
586 indicatorValues = {row[6]:row[7]}
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
587 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
588 else:
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
589 indicatorValues[row[6]] = row[7]
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
590 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
591 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
592 printDBError(error)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
593 return []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
594 connection.close()
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
595 return interactions
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
596 # load first and last object instants
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
597 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF where P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
598
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
599 def createBoundingBoxTable(filename, invHomography = None):
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
600 '''Create the table to store the object bounding boxes in image space
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
601 '''
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
602 connection = sqlite3.connect(filename)
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
603 cursor = connection.cursor()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
604 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
605 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
606 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
607 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from '
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
608 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF where P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
609 'GROUP BY object_id, frame_number')
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
610 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
611 printDBError(error)
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
612 connection.commit()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
613 connection.close()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
614
586
ff4f0ce46ca6 modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 585
diff changeset
615 def loadBoundingBoxTableForDisplay(filename):
647
458890c0437c cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 646
diff changeset
616 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
617 connection = sqlite3.connect(filename)
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
618 cursor = connection.cursor()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
619 boundingBoxes = {} # list of bounding boxes for each instant
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
620 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
621 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
622 result = [row for row in cursor]
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
623 if len(result) > 0:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
624 cursor.execute('SELECT * FROM bounding_boxes')
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
625 for row in cursor:
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
626 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
627 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
628 printDBError(error)
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
629 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
630 connection.close()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
631 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
632
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
633 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
634 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
635 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
636
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
637 def openCheck(filename, option = 'r', quitting = False):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
638 '''Open file filename in read mode by default
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
639 and checks it is open'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
640 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
641 return open(filename, option)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
642 except IOError:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
643 print 'File %s could not be opened.' % filename
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
644 if quitting:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
645 from sys import exit
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
646 exit()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
647 return None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
648
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
649 def readline(f, commentCharacters = commentChar):
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
650 '''Modified readline function to skip comments
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
651 Can take a list of characters or a string (in will work in both)'''
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
652 s = f.readline()
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
653 while (len(s) > 0) and s[0] in commentCharacters:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
654 s = f.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
655 return s.strip()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
656
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
657 def getLines(f, commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
658 '''Gets a complete entry (all the lines) in between delimiterChar.'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
659 dataStrings = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
660 s = readline(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
661 while len(s) > 0:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
662 dataStrings += [s.strip()]
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
663 s = readline(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
664 return dataStrings
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
665
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
666 def writeList(filename, l):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
667 f = openCheck(filename, 'w')
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
668 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
669 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
670 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
671
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
672 def loadListStrings(filename, commentCharacters = commentChar):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
673 f = openCheck(filename, 'r')
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
674 result = getLines(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
675 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
676 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
677
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
678 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
679 values = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
680 for l in loadListStrings(filename, commentCharacters):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
681 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
682 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
683 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
684
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
685 class FakeSecHead(object):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
686 '''Add fake section header [asection]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
687
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
688 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
689 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
690 '''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
691 def __init__(self, fp):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
692 self.fp = fp
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
693 self.sechead = '[main]\n'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
694
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
695 def readline(self):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
696 if self.sechead:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
697 try: return self.sechead
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
698 finally: self.sechead = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
699 else: return self.fp.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
700
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
701 def generatePDLaneColumn(data):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
702 data['LANE'] = data['LANE\LINK\NO'].astype(str)+'_'+data['LANE\INDEX'].astype(str)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
703
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
704 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, nObjects = -1, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
705 '''Reads data from VISSIM .fzp trajectory file
527
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
706 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
707 for example, there seems to be 5 simulation steps per simulated second in VISSIM,
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
708 so simulationStepsPerTimeUnit should be 5,
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
709 so that all times correspond to the number of the simulation step (and can be stored as integers)
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
710
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
711 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
712 (if the object has no such position, it won't be loaded)
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
713
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
714 Assumed to be sorted over time'''
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
715 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
716
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
717 if usePandas:
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
718 from pandas import read_csv
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
719 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
720 generatePDLaneColumn(data)
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
721 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
722 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
723 data = data[data['TIME']>=warmUpLastInstant]
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
724 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
725 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
726 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
727 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
728 tmp = data[data['NO'] == objNum]
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
729 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']))
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
730 # positions should be rounded to nDecimals decimals only
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
731 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
732 if nObjects > 0 and len(objects) >= nObjects:
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
733 break
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
734 return objects.values()
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
735 else:
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
736 inputfile = openCheck(filename, quitting = True)
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
737 # data = pd.read_csv(filename, skiprows=15, delimiter=';')
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
738 # skip header: 15 lines + 1
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
739 line = readline(inputfile, '*$')
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
740 while len(line) > 0:#for line in inputfile:
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
741 data = line.strip().split(';')
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
742 objNum = int(data[1])
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
743 instant = int(float(data[0])*simulationStepsPerTimeUnit)
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
744 s = float(data[4])
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
745 y = float(data[5])
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
746 lane = data[2]+'_'+data[3]
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
747 if objNum not in objects:
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
748 if warmUpLastInstant is None or instant >= warmUpLastInstant:
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
749 if nObjects < 0 or len(objects) < nObjects:
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
750 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
751 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
752 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
753 objects[objNum].timeInterval.last = instant
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
754 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
755 line = readline(inputfile, '*$')
524
1dced8932b08 corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 514
diff changeset
756
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
757 return objects.values()
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
758
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
759 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
760 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
761
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
762 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
763 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
764 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
765 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
766 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
767 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
768 indices = indices | (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
769 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
770 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
771 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
772
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
773 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
774 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
775 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
776
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
777 Vehicles are considered finally stationary
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
778 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
779 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
780 (format as string x_y where x is link index and y is lane index)'''
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
781 from pandas import read_csv
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
782 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
783 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
784 columns += ['LANE\LINK\NO', 'LANE\INDEX']
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
785 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
786 data = selectPDLanes(data, lanes)
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
787 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
788
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
789 nStationary = 0
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
790 nVehicles = 0
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
791 for name, group in data.groupby(['NO'], sort = False):
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
792 nVehicles += 1
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
793 positions = array(group['POS'])
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
794 diff = positions[1:]-positions[:-1]
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
795 if npsum(diff == 0.) >= proportionStationaryTime*len(positions):
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
796 nStationary += 1
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
797
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
798 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
799
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
800 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2):
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
801 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
802
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
803 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
804 one checks when the sign of the position difference inverts
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
805 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
806 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
807 (format as string x_y where x is link index and y is lane index)'''
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
808 from pandas import read_csv, merge
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
809 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
810 data = selectPDLanes(data, lanes)
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
811 merged = merge(data, data, how='inner', left_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], sort = False)
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
812 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
813
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
814 nCollisions = 0
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
815 for name, group in merged.groupby(['LANE\LINK\NO', 'LANE\INDEX', 'NO_x', 'NO_y']):
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
816 diff = group['POS_x'].convert_objects(convert_numeric=True)-group['POS_y'].convert_objects(convert_numeric=True)
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
817 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
818 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
819 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
820 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
821 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
822 nCollisions += 1
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
823 return nCollisions
524
1dced8932b08 corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 514
diff changeset
824
173
319a04ba9033 function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 143
diff changeset
825 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
826 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
827 and returns the list of Feature objects'''
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
828 objects = []
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
829
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
830 inputfile = openCheck(filename, quitting = True)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
831
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
832 def createObject(numbers):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
833 firstFrameNum = int(numbers[1])
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
834 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
835
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
836 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
837 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
838 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
839 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
840 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
841 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
842 userType = int(numbers[10]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
843 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
844 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
845 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
846 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
847 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
848 obj.timeHeadways = [float(numbers[17])] # seconds
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
849 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
850 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
851 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
852 return obj
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
853
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
854 numbers = readline(inputfile).strip().split()
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
855 if (len(numbers) > 0):
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
856 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
857
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
858 for line in inputfile:
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
859 numbers = line.strip().split()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
860 if obj.getNum() != int(numbers[0]):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
861 # check and adapt the length to deal with issues in NGSIM data
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
862 if (obj.length() != obj.positions.length()):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
863 print 'length pb with object %s (%d,%d)' % (obj.getNum(),obj.length(),obj.positions.length())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
864 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
865 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
866 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
867 if (nObjects>0) and (len(objects)>=nObjects):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
868 break
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
869 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
870 else:
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
871 obj.laneNums.append(int(numbers[13]))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
872 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542
a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 541
diff changeset
873 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
874 obj.speeds.append(float(numbers[11]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
875 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
876 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
877 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
878 obj.timeHeadways.append(float(numbers[17]))
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
879
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
880 if (obj.size[0] != float(numbers[8])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
881 print 'changed length obj %d' % (obj.getNum())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
882 if (obj.size[1] != float(numbers[9])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
883 print 'changed width obj %d' % (obj.getNum())
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
884
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
885 inputfile.close()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
886 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
887
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
888 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
889 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
890 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
891 if append:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
892 out = openCheck(outputfile,'a')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
893 else:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
894 out = openCheck(outputfile,'w')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
895 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
896
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
897 features = loadNgsimFile(inputfile, sequenceNum)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
898 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
899 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
900 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
901
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
902 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
903
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
904 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
905
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
906 def writePositionsToCsv(f, obj):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
907 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
908 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
909 curvilinearPositions = obj.getCurvilinearPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
910 for i in xrange(int(obj.length())):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
911 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
912 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
913 if curvilinearPositions is not None:
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
914 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
915 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
916 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
917
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
918 def writeTrajectoriesToCsv(filename, objects):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
919 f = openCheck(filename, 'w')
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
920 for i,obj in enumerate(objects):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
921 writePositionsToCsv(f, obj)
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
922 f.close()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
923
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
924
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
925 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
926 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
927 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
928
664
455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 660
diff changeset
929 class ProcessParameters(VideoFilenameAddable):
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
930 '''Class for all parameters controlling data processing: input,
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
931 method parameters, etc. for tracking, classification and safety
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
932
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
933 Note: framerate is already taken into account'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
934
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
935 def loadConfigFile(self, filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
936 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
937 from os import path
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
938
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
939 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
940 config.readfp(FakeSecHead(openCheck(filename)))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
941 self.sectionHeader = config.sections()[0]
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
942 # Tracking/display parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
943 self.videoFilename = config.get(self.sectionHeader, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
944 self.databaseFilename = config.get(self.sectionHeader, 'database-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
945 self.homographyFilename = config.get(self.sectionHeader, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
946 if (path.exists(self.homographyFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
947 self.homography = loadtxt(self.homographyFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
948 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
949 self.homography = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
950 self.intrinsicCameraFilename = config.get(self.sectionHeader, 'intrinsic-camera-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
951 if (path.exists(self.intrinsicCameraFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
952 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
953 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
954 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
955 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
956 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
957 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
958 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
959 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
960 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
377
2aed569f39e7 added utils
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 344
diff changeset
961
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
962 # Classification parameters
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
963 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
964 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
965 self.speedAggregationQuantile = config.getint(self.sectionHeader, 'speed-aggregation-quantile')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
966 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
967 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
968 self.pedBikeCarSVMFilename = config.get(self.sectionHeader, 'pbv-svm-filename')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
969 self.bikeCarSVMFilename = config.get(self.sectionHeader, 'bv-svm-filename')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
970 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
971 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
972 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
973 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
974 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
975 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
976 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
977 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
978
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
979 # Safety parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
980 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
981 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
982 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
983 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
984 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
985 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
986 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
987 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
988 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
989 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
990 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
991 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
992
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
993 def __init__(self, filename = None):
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
994 if filename is not None:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
995 self.loadConfigFile(filename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
996
680
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
997 def convertToFrames(self, speedRatio = 3.6):
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
998 '''Converts parameters with a relationship to time in 'native' frame time
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
999 speedRatio is the conversion from the speed unit in the config file
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1000 to the distance per second
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1001
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1002 ie param(config file) = speedRatio x fps x param(used in program)
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1003 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1004 denominator = self.videoFrameRate*speedRatio
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1005 denominator2 = denominator**2
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1006 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1007 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1008 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1009 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1010 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1011 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1012 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1013 # special case for the lognormal distribution
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1014 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1015 #self.scaleCyclistSpeed = self.scaleCyclistSpeed
da1352b89d02 classification is working
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 665
diff changeset
1016
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
1017 class SceneParameters(object):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1018 def __init__(self, config, sectionName):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1019 from ConfigParser import NoOptionError
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1020 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1021 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1022 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1023 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1024 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1025 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1026 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1027 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1028 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1029 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1030 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1031 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1032 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1033 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1034 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1035
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1036 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1037 def loadConfigFile(filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1038 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1039 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1040 config.readfp(openCheck(filename))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1041 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1042 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1043 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1044 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1045
382
ba813f148ade development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 377
diff changeset
1046
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1047 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1048 import doctest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1049 import unittest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1050 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1051 unittest.TextTestRunner().run(suite)
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1052 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1053 # #doctest.testfile("example.txt")