repo/traffic-intelligence: python/storage.py annotate

annotate python/storage.py @ 778:bd684e57c431 dev

integrated code from Laurent Gauthier

author	Nicolas Saunier <nicolas.saunier@polymtl.ca>
date	Fri, 05 Feb 2016 17:54:38 -0500
parents	ef6dd60be2e1
children	1b22d81ef5ff

rev	line source
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1 #! /usr/bin/env python
208 d9855499fc88 Added functions to write trajectories through sqlite Francois Belisle <belisle.francois@gmail.com> parents: 204 diff changeset	2 # -- coding: utf-8 --
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	4
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	5 import utils, moving, events, indicators, shutil
665 15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 664 diff changeset	6 from base import VideoFilenameAddable
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	7
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	8 import sqlite3, logging
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	9 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	10 from pandas import read_csv, merge
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	11
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	12
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	13 commentChar = '#'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	14
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	15 delimiterChar = '%';
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	16
50 7b06d649122b re-arrangement Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 48 diff changeset	17 ngsimUserTypes = {'twowheels':1,
204 966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	18 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	19 'truck':3}
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	20
259 8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	21 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	22 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	23 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	24
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	25 # utils
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	26 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	27 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	28
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	29 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	30 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	31 try:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	32 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	33 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	34 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	35 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	36 printDBError(error)
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	37
740 867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	38 def tableExists(filename, tableName):
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	39 'indicates if the table exists in the database'
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	40 try:
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	41 connection = sqlite3.connect(filename)
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	42 cursor = connection.cursor()
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	43 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'')
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	44 return cursor.fetchone()[0] == 1
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	45 except sqlite3.OperationalError as error:
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	46 printDBError(error)
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	47
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	48 def createIndex(connection, tableName, columnName, unique = False):
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	49 '''Creates an index for the column in the table
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	50 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	51 try:
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	52 #connection = sqlite3.connect(filename)
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	53 cursor = connection.cursor()
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	54 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	55 if unique:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	56 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	57 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	58 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	59 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	60 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	61 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	62
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	63 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	64 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	65 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	66 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	67 try:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	68 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	69 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	70 cursor.execute("SELECT COUNT(*) from "+tableName)
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	71 else:
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	72 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	73 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	74 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	75 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	76
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	77 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	78 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	79 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	80 try:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	81 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	82 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	83 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	84 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	85 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	86 else:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	87 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	88 return cursor.fetchone()[0]
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	89 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	90 printDBError(error)
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	91
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	92 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	93 """
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	94 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	95 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	96 """
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	97 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	98
211 ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	99 connection = sqlite3.connect(filename)
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	100 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	101
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	102 try:
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	103 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	104 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	105 printDBError(error)
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	106 return []
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	107
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	108 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	109 matched_indexes.append((row[0],row[1]))
211 ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	110
ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	111 connection.close()
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	112 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	113
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	114 def getObjectCriteria(objectNumbers):
585 aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	115 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	116 query = ''
aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	117 elif type(objectNumbers) == int:
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	118 query = 'between 0 and {0}'.format(objectNumbers-1)
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	119 elif type(objectNumbers) == list:
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	120 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	121 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	122 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	123 query = ''
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	124 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	125
585 aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	126 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None):
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	127 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	128 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	129
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	130 returns a moving object'''
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	131 cursor = connection.cursor()
204 966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	132
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	133 try:
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	134 objectCriteria = getObjectCriteria(objectNumbers)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	135 queryStatement = None
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	136 if trajectoryType == 'feature':
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	137 queryStatement = 'SELECT * from '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	138 if objectNumbers is not None:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	139 queryStatement += ' WHERE trajectory_id '+objectCriteria
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	140 queryStatement += ' ORDER BY trajectory_id, frame_number'
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	141 elif trajectoryType == 'object':
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	142 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	143 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	144 queryStatement += ' and OF.object_id '+objectCriteria
712 21aeadcfbabb added script to test SQLite performance and impact of indices Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 711 diff changeset	145 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	146 elif trajectoryType in ['bbtop', 'bbbottom']:
587 cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	147 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	148 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	149 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	150 corner = 'bottom_right'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	151 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	152 if objectNumbers is not None:
712 21aeadcfbabb added script to test SQLite performance and impact of indices Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 711 diff changeset	153 queryStatement += ' WHERE object_id '+objectCriteria
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	154 queryStatement += ' ORDER BY object_id, frame_number'
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	155 else:
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	156 print('Unknown trajectory type {}'.format(trajectoryType))
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	157 if queryStatement is not None:
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	158 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	159 logging.debug(queryStatement)
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	160 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	161 printDBError(error)
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	162 return []
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	163
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	164 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	165 obj = None
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	166 objects = []
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	167 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	168 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	169 objId = row[0]
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	170 if obj is not None and obj.length() == obj.positions.length():
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	171 objects.append(obj)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	172 elif obj is not None:
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	173 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	174 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	175 else:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	176 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	177 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	178
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	179 if obj is not None and obj.length() == obj.positions.length():
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	180 objects.append(obj)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	181 elif obj is not None:
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	182 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	183
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	184 return objects
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	185
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	186 def loadUserTypesFromTable(cursor, trajectoryType, objectNumbers):
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	187 objectCriteria = getObjectCriteria(objectNumbers)
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	188 queryStatement = 'SELECT object_id, road_user_type from objects'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	189 if objectNumbers is not None:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	190 queryStatement += ' WHERE object_id '+objectCriteria
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	191 cursor.execute(queryStatement)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	192 userTypes = {}
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	193 for row in cursor:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	194 userTypes[row[0]] = row[1]
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	195 return userTypes
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	196
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	197 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False):
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	198 '''Loads the trajectories (in the general sense,
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	199 either features, objects (feature groups) or bounding box series)
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	200 The number loaded is either the first objectNumbers objects,
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	201 or the indices in objectNumbers from the database'''
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	202 connection = sqlite3.connect(filename)
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	203
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	204 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	205 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	206
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	207 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	208 for o,v in zip(objects, objectVelocities):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	209 if o.getNum() == v.getNum():
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	210 o.velocities = v.positions
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	211 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	212 else:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	213 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	214
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	215 if trajectoryType == 'object':
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	216 cursor = connection.cursor()
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	217 try:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	218 # attribute feature numbers to objects
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	219 objectCriteria = getObjectCriteria(objectNumbers)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	220 queryStatement = 'SELECT P.trajectory_id, OF.object_id from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	221 if objectNumbers is not None:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	222 queryStatement += ' and OF.object_id '+objectCriteria
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	223 queryStatement += ' group by P.trajectory_id order by OF.object_id' # order is important to group all features per object
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	224 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	225 logging.debug(queryStatement)
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	226
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	227 featureNumbers = {}
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	228 for row in cursor:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	229 objId = row[1]
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	230 if objId not in featureNumbers:
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	231 featureNumbers[objId] = [row[0]]
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	232 else:
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	233 featureNumbers[objId].append(row[0])
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	234
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	235 for obj in objects:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	236 obj.featureNumbers = featureNumbers[obj.getNum()]
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	237
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	238 # load userType
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	239 userTypes = loadUserTypesFromTable(cursor, trajectoryType, objectNumbers)
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	240 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	241 obj.userType = userTypes[obj.getNum()]
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	242
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	243 if withFeatures:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	244 nFeatures = 0
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	245 for obj in objects:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	246 nFeatures = max(nFeatures, max(obj.featureNumbers))
702 f0a897d7f3a5 corrected bug when loading features with objects Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 695 diff changeset	247 features = loadTrajectoriesFromSqlite(filename, 'feature', nFeatures+1)
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	248 for obj in objects:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	249 obj.setFeatures(features)
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	250
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	251 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	252 printDBError(error)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	253 objects = []
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	254
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	255 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	256 return objects
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	257
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	258 def loadCurvilinearTrajectoriesFromSqlite(filename, objects, objectNumbers = None):
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	259 '''load alignement curvilinear positions (s_coordinate, y_coordinate, lane)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	260 from an object database to an existing MovingObject list'''
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	261 connection = sqlite3.connect(inputFilename)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	262 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	263
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	264 try:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	265 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number')
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	266 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	267 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	268 return []
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	269
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	270 objNum = None
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	271 for row in cursor:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	272 if objNum != row[0]:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	273 objNum = row[0]
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	274 objects[objNum].curvilinearPositions = moving.CurvilinearPositions()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	275 else:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	276 objects[objNum].curvilinearPositions.addPositionsSYL(row[2],row[3],row[4])
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	277
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	278 return objects
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	279
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	280 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType, withFeatures = False):
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	281 '''Writes features, ie the trajectories positions (and velocities if exist)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	282 with their instants to a specified sqlite file
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	283
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	284 TODO: Not implemented for other trajectoryType than features
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	285 For objects, with features will control whether the features
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	286 corresponding to the object are also saved'''
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	287
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	288 connection = sqlite3.connect(outputFilename)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	289 try:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	290 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	291
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	292 if trajectoryType in ['feature', 'curvilinear']:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	293 cursor.execute("CREATE TABLE IF NOT EXISTS positions (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	294 cursor.execute("CREATE TABLE IF NOT EXISTS velocities (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	295 if trajectoryType == 'curvilinear':
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	296 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))")
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	297
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	298 positionQuery = "insert into positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	299 velocityQuery = "insert into velocities (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	300 curvilinearQuery = "insert into curvilinear_positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	301 for obj in objects:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	302 num = obj.getNum()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	303 frame_number = obj.getFirstInstant()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	304 for position in obj.getPositions():
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	305 cursor.execute(positionQuery, (num, frame_number, position.x, position.y))
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	306 frame_number += 1
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	307 # velocities
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	308 velocities = obj.getVelocities()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	309 if velocities is not None:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	310 frame_number = obj.getFirstInstant()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	311 for i in xrange(velocities.length()-1):
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	312 v = velocities[i]
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	313 cursor.execute(velocityQuery, (num, frame_number, v.x, v.y))
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	314 frame_number += 1
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	315 # curvilinear trajectories
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	316 if trajectoryType == 'curvilinear' and hasattr(obj, 'curvilinearPositions') is not None:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	317 frame_number = obj.getFirstInstant()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	318 for position in obj.getCurvilinearPositions():
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	319 cursor.execute(curvilinearQuery, (num, frame_number, position[0], position[1], position[2]))
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	320 frame_number += 1
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	321 connection.commit()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	322 #elif trajectoryType == 'object':
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	323 else:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	324 print('Unknown trajectory type {}'.format(trajectoryType))
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	325 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	326 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	327 connection.close()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	328
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	329 def savePrototypesToSqlite(filename, prototypes, trajectoryType = 'feature'):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	330 'Work in progress, do not use'
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	331 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	332 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	333 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	334 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (id INTEGER PRIMARY KEY, object_id INTEGER, trajectory_id INTEGER, nMatchings INTEGER, FOREIGN KEY(object_id) REFERENCES objects(id), FOREIGN KEY(trajectory_id) REFERENCES positions(trajectory_id))')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	335 #for inter in interactions:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	336 # saveInteraction(cursor, inter)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	337 except sqlite3.OperationalError as error:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	338 printDBError(error)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	339 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	340 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	341
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	342 def loadPrototypesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	343 pass
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	344
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	345 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None):
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	346 '''Loads bounding box moving object from an SQLite
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	347 (format of SQLite output by the ground truth annotation tool
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	348 or Urban Tracker
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	349
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	350 Load descriptions?'''
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	351 connection = sqlite3.connect(filename)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	352 objects = []
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	353
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	354 if objectType == 'bb':
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	355 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	356 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	357 userTypes = loadUserTypesFromTable(connection.cursor(), 'object', objectNumbers) # string format is same as object
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	358
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	359 for t, b in zip(topCorners, bottomCorners):
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	360 num = t.getNum()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	361 if t.getNum() == b.getNum():
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	362 annotation = moving.BBMovingObject(num, t.getTimeInterval(), t, b, userTypes[num])
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	363 objects.append(annotation)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	364 else:
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	365 print ('Unknown type of bounding box {}'.format(objectType))
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	366
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	367 connection.close()
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	368 return objects
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	369
584 5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 564 diff changeset	370 def deleteFromSqlite(filename, dataType):
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 564 diff changeset	371 'Deletes (drops) some tables in the filename depending on type of data'
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	372 import os
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	373 if os.path.isfile(filename):
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	374 connection = sqlite3.connect(filename)
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	375 if dataType == 'object':
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	376 dropTables(connection, ['objects', 'objects_features'])
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	377 elif dataType == 'interaction':
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	378 dropTables(connection, ['interactions', 'indicators'])
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	379 elif dataType == 'bb':
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	380 dropTables(connection, ['bounding_boxes'])
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	381 else:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	382 print('Unknown data type {} to delete from database'.format(dataType))
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	383 connection.close()
344 14a2405f54f8 slight modification to safety analysis and generalized script to delete computed data (objects and interactions) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 343 diff changeset	384 else:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	385 print('{} does not exist'.format(filename))
235 584613399513 added script and functions to remove object tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 218 diff changeset	386
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	387 def createInteractionTable(cursor):
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	388 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	389
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	390 def createIndicatorTables(cursor):
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	391 # cursor.execute('CREATE TABLE IF NOT EXISTS indicators (id INTEGER PRIMARY KEY, interaction_id INTEGER, indicator_type INTEGER, FOREIGN KEY(interaction_id) REFERENCES interactions(id))')
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	392 # cursor.execute('CREATE TABLE IF NOT EXISTS indicator_values (indicator_id INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(indicator_id) REFERENCES indicators(id), PRIMARY KEY(indicator_id, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	393 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	394
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	395 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	396 roadUserNumbers = list(interaction.getRoadUserNumbers())
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	397 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	398
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	399 def saveInteractions(filename, interactions):
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	400 'Saves the interactions in the table'
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	401 connection = sqlite3.connect(filename)
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	402 cursor = connection.cursor()
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	403 try:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	404 createInteractionTable(cursor)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	405 for inter in interactions:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	406 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	407 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	408 printDBError(error)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	409 connection.commit()
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	410 connection.close()
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	411
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	412 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	413 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	414 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	415 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	416
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	417 def saveIndicators(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	418 'Saves the indicator values in the table'
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	419 connection = sqlite3.connect(filename)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	420 cursor = connection.cursor()
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	421 try:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	422 createInteractionTable(cursor)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	423 createIndicatorTables(cursor)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	424 for inter in interactions:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	425 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	426 for indicatorName in indicatorNames:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	427 indicator = inter.getIndicator(indicatorName)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	428 if indicator is not None:
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	429 saveIndicator(cursor, inter.getNum(), indicator)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	430 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	431 printDBError(error)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	432 connection.commit()
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	433 connection.close()
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	434
482 f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	435 def loadInteractions(filename):
f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	436 '''Loads interaction and their indicators
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	437
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	438 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	439 interactions = []
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	440 connection = sqlite3.connect(filename)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	441 cursor = connection.cursor()
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	442 try:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	443 cursor.execute('select INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	444 interactionNum = -1
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	445 indicatorTypeNum = -1
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	446 tmpIndicators = {}
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	447 for row in cursor:
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	448 if row[0] != interactionNum:
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	449 interactionNum = row[0]
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	450 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	451 interactions[-1].indicators = {}
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	452 if indicatorTypeNum != row[5] or row[0] != interactionNum:
695 957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 693 diff changeset	453 indicatorTypeNum = row[5]
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	454 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	455 indicatorValues = {row[6]:row[7]}
695 957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 693 diff changeset	456 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators)
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	457 else:
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	458 indicatorValues[row[6]] = row[7]
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	459 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	460 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	461 printDBError(error)
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	462 return []
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	463 connection.close()
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	464 return interactions
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	465 # load first and last object instants
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	466 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	467
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	468 def createBoundingBoxTable(filename, invHomography = None):
482 f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	469 '''Create the table to store the object bounding boxes in image space
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	470 '''
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	471 connection = sqlite3.connect(filename)
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	472 cursor = connection.cursor()
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	473 try:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	474 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	475 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	476 '(SELECT object_id, frame_number, (x{}+y{}+{})/w as x, (x{}+y{}+{})/w as y from '
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	477 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate{}+P.y_coordinate{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	478 'GROUP BY object_id, frame_number')
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	479 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	480 printDBError(error)
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	481 connection.commit()
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	482 connection.close()
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	483
586 ff4f0ce46ca6 modified name for loading bounding boxes (only for display) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 585 diff changeset	484 def loadBoundingBoxTableForDisplay(filename):
647 458890c0437c cleaned functions to lead bounding boxes (for display) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 646 diff changeset	485 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	486 connection = sqlite3.connect(filename)
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	487 cursor = connection.cursor()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	488 boundingBoxes = {} # list of bounding boxes for each instant
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	489 try:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	490 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	491 result = cursor.fetchall()
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	492 if len(result) > 0:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	493 cursor.execute('SELECT * FROM bounding_boxes')
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	494 for row in cursor:
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	495 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	496 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	497 printDBError(error)
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	498 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	499 connection.close()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	500 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	501
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	502 #########################
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	503 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	504 #########################
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	505
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	506 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	507 """ prototype dataset is a dictionary with keys== routes, values== prototypes Ids """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	508 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	509 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	510
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	511 cursor.execute("CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	512
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	513 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	514 if prototypes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	515 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	516 cursor.execute("insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)",(i,route[0],route[1],nMatching[route][i]))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	517
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	518 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	519 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	520
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	521 def readPrototypesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	522 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	523 This function loads the prototype file in the database
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	524 It returns a dictionary for prototypes for each route and nMatching
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	525 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	526 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	527 nMatching={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	528
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	529 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	530 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	531
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	532 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	533 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	534 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	535 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	536 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	537
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	538 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	539 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	540 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	541 prototypes[route]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	542 prototypes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	543 nMatching[row[0]]=row[3]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	544
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	545 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	546 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	547
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	548 def writeLabelsToSqlite(labels, outputFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	549 """ labels is a dictionary with keys: routes, values: prototypes Ids
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	550 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	551 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	552 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	553
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	554 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	555
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	556 for route in labels.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	557 if labels[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	558 for i in labels[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	559 for j in labels[route][i]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	560 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	561
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	562 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	563 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	564
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	565 def loadLabelsFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	566 labels = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	567
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	568 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	569 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	570
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	571 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	572 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	573 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	574 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	575 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	576
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	577 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	578 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	579 p=row[3]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	580 if route not in labels.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	581 labels[route]={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	582 if p not in labels[route].keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	583 labels[route][p]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	584 labels[route][p].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	585
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	586 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	587 return labels
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	588
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	589 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	590 """ to match the format of second layer prototypes"""
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	591 connection = sqlite3.connect(outFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	592 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	593
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	594 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	595
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	596 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	597 if prototypes[route]!={}:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	598 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	599 if prototypes[route][i]!= []:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	600 for j in prototypes[route][i]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	601 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	602
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	603 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	604 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	605
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	606 def loadSpeedPrototypeFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	607 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	608 This function loads the prototypes table in the database of name <filename>.
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	609 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	610 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	611 nMatching={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	612 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	613 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	614
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	615 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	616 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	617 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	618 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	619 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	620
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	621 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	622 route=(row[2],row[3])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	623 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	624 prototypes[route]={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	625 if row[1] not in prototypes[route].keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	626 prototypes[route][row[1]]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	627 prototypes[route][row[1]].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	628 nMatching[row[0]]=row[4]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	629
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	630 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	631 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	632
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	633
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	634 def writeRoutesToSqlite(Routes, outputFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	635 """ This function writes the activity path define by start and end IDs"""
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	636 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	637 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	638
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	639 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	640
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	641 for route in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	642 if Routes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	643 for i in Routes[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	644 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	645
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	646 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	647 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	648
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	649 def loadRoutesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	650 Routes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	651
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	652 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	653 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	654
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	655 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	656 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	657 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	658 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	659 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	660
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	661 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	662 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	663 if route not in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	664 Routes[route]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	665 Routes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	666
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	667 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	668 return Routes
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	669
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	670 def setRoutes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	671 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	672 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	673 for obj in objects:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	674 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum()))
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	675 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum()))
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	676 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	677 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	678
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	679 def setRoadUserTypes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	680 '''Saves the user types of the objects in the sqlite database stored in filename
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	681 The objects should exist in the objects table'''
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	682 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	683 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	684 for obj in objects:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	685 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum()))
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	686 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	687 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	688
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	689 #########################
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	690 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	691 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	692
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	693 def openCheck(filename, option = 'r', quitting = False):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	694 '''Open file filename in read mode by default
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	695 and checks it is open'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	696 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	697 return open(filename, option)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	698 except IOError:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	699 print 'File %s could not be opened.' % filename
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	700 if quitting:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	701 from sys import exit
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	702 exit()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	703 return None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	704
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	705 def readline(f, commentCharacters = commentChar):
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	706 '''Modified readline function to skip comments
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	707 Can take a list of characters or a string (in will work in both)'''
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	708 s = f.readline()
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	709 while (len(s) > 0) and s[0] in commentCharacters:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	710 s = f.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	711 return s.strip()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	712
722 49e99ca34a7d corrected bugs in old ubc code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 718 diff changeset	713 def getLines(f, delimiterChar = delimiterChar, commentCharacters = commentChar):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	714 '''Gets a complete entry (all the lines) in between delimiterChar.'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	715 dataStrings = []
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	716 s = readline(f, commentCharacters)
722 49e99ca34a7d corrected bugs in old ubc code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 718 diff changeset	717 while len(s) > 0 and s[0] != delimiterChar:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	718 dataStrings += [s.strip()]
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	719 s = readline(f, commentCharacters)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	720 return dataStrings
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	721
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	722 def writeList(filename, l):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	723 f = openCheck(filename, 'w')
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	724 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	725 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	726 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	727
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	728 def loadListStrings(filename, commentCharacters = commentChar):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	729 f = openCheck(filename, 'r')
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	730 result = getLines(f, commentCharacters)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	731 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	732 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	733
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	734 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = commentChar):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	735 values = []
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	736 for l in loadListStrings(filename, commentCharacters):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	737 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	738 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	739 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	740
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	741 class FakeSecHead(object):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	742 '''Add fake section header [asection]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	743
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	744 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	745 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	746 '''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	747 def __init__(self, fp):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	748 self.fp = fp
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	749 self.sechead = '[main]\n'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	750
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	751 def readline(self):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	752 if self.sechead:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	753 try: return self.sechead
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	754 finally: self.sechead = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	755 else: return self.fp.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	756
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	757 def generatePDLaneColumn(data):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	758 data['LANE'] = data['LANE\LINK\NO'].astype(str)+'_'+data['LANE\INDEX'].astype(str)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	759
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	760 def convertTrajectoriesVissimToSqlite(filename):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	761 '''Relies on a system call to sqlite3
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	762 sqlite3 [file.sqlite] < import_fzp.sql'''
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	763 sqlScriptFilename = "import_fzp.sql"
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	764 # create sql file
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	765 out = openCheck(sqlScriptFilename, "w")
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	766 out.write(".separator \";\"\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	767 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	768 ".import "+filename+" curvilinear_positions\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	769 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n")
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	770 out.close()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	771 # system call
753 3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	772 from subprocess import check_call
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	773 out = openCheck("err.log", "w")
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	774 check_call("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out, shell = True)
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	775 out.close()
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	776 shutil.os.remove(sqlScriptFilename)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	777
754 782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	778 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds):
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	779 '''Finds the ids of the objects that go through any of the link in the list linkIds'''
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	780 connection = sqlite3.connect(filename)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	781 cursor = connection.cursor()
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	782 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')'
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	783 try:
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	784 cursor.execute(queryStatement)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	785 return [row[0] for row in cursor]
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	786 except sqlite3.OperationalError as error:
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	787 printDBError(error)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	788
759 a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	789 def getNObjectsInLinkFromVissimFile(filename, linkIds):
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	790 '''Returns the number of objects that traveled through the link ids'''
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	791 connection = sqlite3.connect(filename)
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	792 cursor = connection.cursor()
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	793 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id'
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	794 try:
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	795 cursor.execute(queryStatement)
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	796 return {row[0]:row[1] for row in cursor}
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	797 except sqlite3.OperationalError as error:
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	798 printDBError(error)
754 782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	799
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	800 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	801 '''Reads data from VISSIM .fzp trajectory file
755 f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	802 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second)
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	803 for example, there seems to be 10 simulation steps per simulated second in VISSIM,
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	804 so simulationStepsPerTimeUnit should be 10,
527 37830a831818 loading from VISSIM trajectory data works Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 526 diff changeset	805 so that all times correspond to the number of the simulation step (and can be stored as integers)
660 994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	806
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	807 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	808 (if the object has no such position, it won't be loaded)
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	809
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	810 Assumed to be sorted over time
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	811 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id'''
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	812 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	813
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	814 if usePandas:
718 2cd245cb780d added option to set low_memory = False for pandas.read_csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 714 diff changeset	815 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	816 generatePDLaneColumn(data)
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	817 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660 994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	818 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	819 data = data[data['TIME']>=warmUpLastInstant]
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	820 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	821 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	822 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	823 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	824 tmp = data[data['NO'] == objNum]
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	825 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']))
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	826 # positions should be rounded to nDecimals decimals only
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	827 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	828 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers:
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	829 objects.values()
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	830 else:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	831 if filename.endswith(".fzp"):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	832 inputfile = openCheck(filename, quitting = True)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	833 line = readline(inputfile, '*$')
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	834 while len(line) > 0:#for line in inputfile:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	835 data = line.strip().split(';')
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	836 objNum = int(data[1])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	837 instant = float(data[0])*simulationStepsPerTimeUnit
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	838 s = float(data[4])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	839 y = float(data[5])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	840 lane = data[2]+'_'+data[3]
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	841 if objNum not in objects:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	842 if warmUpLastInstant is None or instant >= warmUpLastInstant:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	843 if objectNumbers is None or len(objects) < objectNumbers:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	844 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	845 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	846 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	847 objects[objNum].timeInterval.last = instant
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	848 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	849 line = readline(inputfile, '*$')
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	850 elif filename.endswith(".sqlite"):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	851 connection = sqlite3.connect(filename)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	852 cursor = connection.cursor()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	853 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions'
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	854 if objectNumbers is not None:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	855 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	856 queryStatement += ' ORDER BY trajectory_id, t'
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	857 try:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	858 cursor.execute(queryStatement)
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	859 for row in cursor:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	860 objNum = row[1]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	861 instant = row[0]*simulationStepsPerTimeUnit
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	862 s = row[4]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	863 y = row[5]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	864 lane = '{}_{}'.format(row[2], row[3])
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	865 if objNum not in objects:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	866 if warmUpLastInstant is None or instant >= warmUpLastInstant:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	867 if objectNumbers is None or len(objects) < objectNumbers:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	868 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	869 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	870 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	871 objects[objNum].timeInterval.last = instant
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	872 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	873 except sqlite3.OperationalError as error:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	874 printDBError(error)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	875 else:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	876 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
751 79405a938407 corrected bug Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 750 diff changeset	877 return objects.values()
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	878
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	879 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	880 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	881
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	882 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	883 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	884 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	885 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	886 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	887 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	888 indices = indices \| (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	889 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	890 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	891 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	892
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	893 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	894 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	895 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	896
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	897 Vehicles are considered finally stationary
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	898 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	899 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	900 (format as string x_y where x is link index and y is lane index)'''
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	901 if filename.endswith(".fzp"):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	902 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	903 if lanes is not None:
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	904 columns += ['LANE\LINK\NO', 'LANE\INDEX']
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	905 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	906 data = selectPDLanes(data, lanes)
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	907 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	908
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	909 nStationary = 0
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	910 nVehicles = 0
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	911 for name, group in data.groupby(['NO'], sort = False):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	912 nVehicles += 1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	913 positions = array(group['POS'])
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	914 diff = positions[1:]-positions[:-1]
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	915 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	916 nStationary += 1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	917 elif filename.endswith(".sqlite"):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	918 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	919 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time?
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	920 pass
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	921 else:
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	922 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	923
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	924 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	925
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	926 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True):
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	927 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	928
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	929 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	930 one checks when the sign of the position difference inverts
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	931 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	932 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	933 (format as string x_y where x is link index and y is lane index)'''
718 2cd245cb780d added option to set low_memory = False for pandas.read_csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 714 diff changeset	934 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	935 data = selectPDLanes(data, lanes)
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	936 data = data.convert_objects(convert_numeric=True)
ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	937
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	938 merged = merge(data, data, how='inner', left_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], sort = False)
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	939 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	940
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	941 nCollisions = 0
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	942 for name, group in merged.groupby(['LANE\LINK\NO', 'LANE\INDEX', 'NO_x', 'NO_y']):
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	943 diff = group['POS_x']-group['POS_y']
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	944 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	945 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	946 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	947 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	948 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	949 nCollisions += 1
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	950
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	951 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by??
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	952 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id;
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	953 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	954 return nCollisions
524 1dced8932b08 corrected bugs Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 514 diff changeset	955
173 319a04ba9033 function name change Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 143 diff changeset	956 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	957 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	958 and returns the list of Feature objects'''
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	959 objects = []
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	960
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	961 inputfile = openCheck(filename, quitting = True)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	962
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	963 def createObject(numbers):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	964 firstFrameNum = int(numbers[1])
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	965 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	966
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	967 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	968 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	969 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	970 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	971 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	972 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	973 userType = int(numbers[10]))
78 99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	974 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	975 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	976 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	977 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	978 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	979 obj.timeHeadways = [float(numbers[17])] # seconds
327 42f2b46ec210 added class for trajectories in curvilinear coordinates Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 326 diff changeset	980 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	981 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	982 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	983 return obj
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	984
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	985 numbers = readline(inputfile).strip().split()
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	986 if (len(numbers) > 0):
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	987 obj = createObject(numbers)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	988
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	989 for line in inputfile:
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	990 numbers = line.strip().split()
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	991 if obj.getNum() != int(numbers[0]):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	992 # check and adapt the length to deal with issues in NGSIM data
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	993 if (obj.length() != obj.positions.length()):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	994 print 'length pb with object %s (%d,%d)' % (obj.getNum(),obj.length(),obj.positions.length())
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	995 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	996 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	997 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	998 if (nObjects>0) and (len(objects)>=nObjects):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	999 break
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1000 obj = createObject(numbers)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1001 else:
327 42f2b46ec210 added class for trajectories in curvilinear coordinates Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 326 diff changeset	1002 obj.laneNums.append(int(numbers[13]))
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1003 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542 a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 541 diff changeset	1004 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1005 obj.speeds.append(float(numbers[11]))
78 99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1006 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1007 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1008 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1009 obj.timeHeadways.append(float(numbers[17]))
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1010
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1011 if (obj.size[0] != float(numbers[8])):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1012 print 'changed length obj %d' % (obj.getNum())
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1013 if (obj.size[1] != float(numbers[9])):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1014 print 'changed width obj %d' % (obj.getNum())
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1015
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1016 inputfile.close()
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1017 return objects
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1018
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1019 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1020 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1021 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1022 if append:
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1023 out = openCheck(outputfile,'a')
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1024 else:
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1025 out = openCheck(outputfile,'w')
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1026 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1027
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1028 features = loadNgsimFile(inputfile, sequenceNum)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1029 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1030 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1031 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1032
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1033 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1034
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1035 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1036
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1037 def writePositionsToCsv(f, obj):
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1038 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1039 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1040 curvilinearPositions = obj.getCurvilinearPositions()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1041 for i in xrange(int(obj.length())):
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1042 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1043 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1044 if curvilinearPositions is not None:
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1045 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1046 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1047 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1048
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1049 def writeTrajectoriesToCsv(filename, objects):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	1050 f = openCheck(filename, 'w')
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1051 for i,obj in enumerate(objects):
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1052 writePositionsToCsv(f, obj)
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1053 f.close()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1054
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1055
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1056 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1057 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1058 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1059
664 455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 660 diff changeset	1060 class ProcessParameters(VideoFilenameAddable):
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1061 '''Class for all parameters controlling data processing: input,
6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1062 method parameters, etc. for tracking, classification and safety
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1063
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1064 Note: framerate is already taken into account'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1065
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1066 def loadConfigFile(self, filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1067 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1068 from os import path
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1069
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1070 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1071 config.readfp(FakeSecHead(openCheck(filename)))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1072 self.sectionHeader = config.sections()[0]
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1073 # Tracking/display parameters
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1074 self.videoFilename = config.get(self.sectionHeader, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1075 self.databaseFilename = config.get(self.sectionHeader, 'database-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1076 self.homographyFilename = config.get(self.sectionHeader, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1077 if (path.exists(self.homographyFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1078 self.homography = loadtxt(self.homographyFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1079 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1080 self.homography = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1081 self.intrinsicCameraFilename = config.get(self.sectionHeader, 'intrinsic-camera-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1082 if (path.exists(self.intrinsicCameraFilename)):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1083 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1084 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1085 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1086 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1087 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1088 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1089 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1090 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1091 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
377 2aed569f39e7 added utils Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 344 diff changeset	1092
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1093 # Classification parameters
680 da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1094 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1095 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1096 self.speedAggregationQuantile = config.getint(self.sectionHeader, 'speed-aggregation-quantile')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1097 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	1098 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
680 da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1099 self.pedBikeCarSVMFilename = config.get(self.sectionHeader, 'pbv-svm-filename')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1100 self.bikeCarSVMFilename = config.get(self.sectionHeader, 'bv-svm-filename')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1101 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1102 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1103 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1104 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1105 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1106 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1107 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1108 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1109
6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1110 # Safety parameters
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1111 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1112 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1113 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1114 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1115 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1116 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1117 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1118 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1119 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1120 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1121 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1122 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1123
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1124 def __init__(self, filename = None):
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1125 if filename is not None:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1126 self.loadConfigFile(filename)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1127
680 da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1128 def convertToFrames(self, speedRatio = 3.6):
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1129 '''Converts parameters with a relationship to time in 'native' frame time
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1130 speedRatio is the conversion from the speed unit in the config file
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1131 to the distance per second
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1132
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1133 ie param(config file) = speedRatio x fps x param(used in program)
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1134 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1135 denominator = self.videoFrameRate*speedRatio
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1136 denominator2 = denominator**2
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1137 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1138 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1139 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1140 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1141 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1142 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1143 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1144 # special case for the lognormal distribution
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1145 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1146 #self.scaleCyclistSpeed = self.scaleCyclistSpeed
da1352b89d02 classification is working Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 665 diff changeset	1147
665 15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 664 diff changeset	1148 class SceneParameters(object):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1149 def __init__(self, config, sectionName):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1150 from ConfigParser import NoOptionError
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1151 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1152 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1153 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1154 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1155 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1156 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1157 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1158 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1159 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1160 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1161 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1162 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1163 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1164 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1165 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1166
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1167 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1168 def loadConfigFile(filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1169 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1170 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1171 config.readfp(openCheck(filename))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1172 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1173 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1174 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1175 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1176
382 ba813f148ade development for clustering Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 377 diff changeset	1177
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1178 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1179 import doctest
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1180 import unittest
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1181 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1182 unittest.TextTestRunner().run(suite)
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1183 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1184 # #doctest.testfile("example.txt")

Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 778:bd684e57c431 dev