repo/traffic-intelligence: python/storage.py annotate

annotate python/storage.py @ 919:7b3f2e0a2652

saving and loading prototype trajectories

author	Nicolas Saunier <nicolas.saunier@polymtl.ca>
date	Wed, 05 Jul 2017 13:16:47 -0400
parents	3a06007a4bb7
children	499154254f37

rev	line source
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1 #! /usr/bin/env python
208 d9855499fc88 Added functions to write trajectories through sqlite Francois Belisle <belisle.francois@gmail.com> parents: 204 diff changeset	2 # -- coding: utf-8 --
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	4
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	5 import utils, moving, events, indicators, shutil
665 15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 664 diff changeset	6 from base import VideoFilenameAddable
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	7
813 ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	8 from os import path
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	9 from copy import copy
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	10 import sqlite3, logging
901 753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	11 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt, floor as npfloor, ceil as npceil, linalg
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	12 from pandas import read_csv, merge
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	13
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	14
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	15 commentChar = '#'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	16
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	17 delimiterChar = '%';
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	18
50 7b06d649122b re-arrangement Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 48 diff changeset	19 ngsimUserTypes = {'twowheels':1,
204 966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	20 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	21 'truck':3}
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	22
908 b297525b2cbf added options to the prototype cluster algorithm, work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 901 diff changeset	23 tableNames = {'feature':'positions',
b297525b2cbf added options to the prototype cluster algorithm, work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 901 diff changeset	24 'object': 'objects',
b297525b2cbf added options to the prototype cluster algorithm, work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 901 diff changeset	25 'objectfeatures': 'positions'}
b297525b2cbf added options to the prototype cluster algorithm, work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 901 diff changeset	26
259 8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	27 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	28 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	29 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 246 diff changeset	30
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	31 # utils
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	32 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	33 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	34
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	35 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	36 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	37 try:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	38 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	39 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	40 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	41 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	42 printDBError(error)
343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	43
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	44 def deleteFromSqlite(filename, dataType):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	45 'Deletes (drops) some tables in the filename depending on type of data'
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	46 if path.isfile(filename):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	47 connection = sqlite3.connect(filename)
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	48 if dataType == 'object':
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	49 dropTables(connection, ['objects', 'objects_features'])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	50 elif dataType == 'interaction':
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	51 dropTables(connection, ['interactions', 'indicators'])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	52 elif dataType == 'bb':
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	53 dropTables(connection, ['bounding_boxes'])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	54 elif dataType == 'pois':
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	55 dropTables(connection, ['gaussians2d', 'objects_pois'])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	56 elif dataType == 'prototype':
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	57 dropTables(connection, ['prototypes', 'prototype_positions', 'prototype_velocities'])
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	58 else:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	59 print('Unknown data type {} to delete from database'.format(dataType))
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	60 connection.close()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	61 else:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	62 print('{} does not exist'.format(filename))
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	63
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	64 def tableExists(connection, tableName):
740 867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	65 'indicates if the table exists in the database'
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	66 try:
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	67 #connection = sqlite3.connect(filename)
740 867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	68 cursor = connection.cursor()
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	69 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'')
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	70 return cursor.fetchone()[0] == 1
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	71 except sqlite3.OperationalError as error:
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	72 printDBError(error)
867bab9f317a function to check the existence of tables Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 736 diff changeset	73
830 2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	74 def createTrajectoryTable(cursor, tableName):
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	75 if tableName.endswith('positions') or tableName.endswith('velocities'):
830 2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	76 cursor.execute("CREATE TABLE IF NOT EXISTS "+tableName+" (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	77 else:
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	78 print('Unallowed name {} for trajectory table'.format(tableName))
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	79
879 f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	80 def createObjectsTable(cursor):
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	81 cursor.execute("CREATE TABLE IF NOT EXISTS objects (object_id INTEGER, road_user_type INTEGER, n_objects INTEGER, PRIMARY KEY(object_id))")
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	82
913 1cd878812529 work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 910 diff changeset	83 def createAssignmentTable(cursor, objectType1, objectType2, columnName1, columnName2):
1cd878812529 work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 910 diff changeset	84 cursor.execute("CREATE TABLE IF NOT EXISTS "+objectType1+"s_"+objectType2+"s ("+columnName1+" INTEGER, "+columnName1+" INTEGER, PRIMARY KEY("+columnName1+","+columnName2+"))")
1cd878812529 work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 910 diff changeset	85
1cd878812529 work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 910 diff changeset	86 def createObjectsFeaturesTable(cursor): # same as
1cd878812529 work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 910 diff changeset	87 cursor.execute("CREATE TABLE IF NOT EXISTS objects_features (object_id INTEGER, trajectory_id INTEGER, PRIMARY KEY(object_id, trajectory_id))")
879 f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	88
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	89
830 2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	90 def createCurvilinearTrajectoryTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	91 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	92
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	93 def createFeatureCorrespondenceTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	94 cursor.execute('CREATE TABLE IF NOT EXISTS feature_correspondences (trajectory_id INTEGER, source_dbname VARCHAR, db_trajectory_id INTEGER, PRIMARY KEY(trajectory_id))')
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	95
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	96 def createInteractionTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	97 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	98
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	99 def createIndicatorTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	100 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	101
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	102 def insertTrajectoryQuery(tableName):
2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	103 return "INSERT INTO "+tableName+" (trajectory_id, frame_number, x_coordinate, y_coordinate) VALUES (?,?,?,?)"
831 a8ff35e6fb43 forgotten change Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 830 diff changeset	104
879 f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	105 def insertObjectQuery():
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	106 return "INSERT INTO objects (object_id, road_user_type, n_objects) VALUES (?,?,?)"
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	107
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	108 def insertObjectFeatureQuery():
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	109 return "INSERT INTO objects_features (object_id, trajectory_id) VALUES (?,?)"
f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	110
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	111 def createIndex(connection, tableName, columnName, unique = False):
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	112 '''Creates an index for the column in the table
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	113 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	114 try:
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	115 #connection = sqlite3.connect(filename)
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	116 cursor = connection.cursor()
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	117 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	118 if unique:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	119 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	120 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	121 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	122 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	123 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	124 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	125
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	126 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	127 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	128 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	129 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	130 try:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	131 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	132 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	133 cursor.execute("SELECT COUNT(*) from "+tableName)
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	134 else:
714 d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	135 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	136 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	137 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	138 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	139
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	140 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	141 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	142 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	143 try:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	144 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	145 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	146 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	147 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	148 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	149 else:
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	150 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 712 diff changeset	151 return cursor.fetchone()[0]
711 523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	152 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	153 printDBError(error)
523eda2fafd4 added function to create index Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 704 diff changeset	154
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	155 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	156 """
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	157 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	158 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	159 """
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	160 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	161
211 ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	162 connection = sqlite3.connect(filename)
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	163 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	164
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	165 try:
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	166 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	167 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	168 printDBError(error)
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	169 return []
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	170
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	171 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	172 matched_indexes.append((row[0],row[1]))
211 ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	173
ada6e8fbe4c6 2 Changes : Francois Belisle <belisle.francois@gmail.com> parents: 209 diff changeset	174 connection.close()
209 746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	175 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches. Francois Belisle <belisle.francois@gmail.com> parents: 208 diff changeset	176
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	177 def getObjectCriteria(objectNumbers):
585 aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	178 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	179 query = ''
aab2242ea88c minor modification of objectNumbers to None Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 584 diff changeset	180 elif type(objectNumbers) == int:
852 45a53542e046 updated unnecessary complicated query Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 850 diff changeset	181 query = '<= {0}'.format(objectNumbers-1)
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	182 elif type(objectNumbers) == list:
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	183 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	184 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	185 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	186 query = ''
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	187 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	188
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	189 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None, timeStep = None):
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	190 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	191 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	192
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	193 returns a moving object'''
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	194 cursor = connection.cursor()
204 966c2cd2bd9f added code to load object trajectories (average of features) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 203 diff changeset	195
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	196 try:
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	197 objectCriteria = getObjectCriteria(objectNumbers)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	198 queryStatement = None
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	199 if trajectoryType == 'feature':
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	200 queryStatement = 'SELECT * from '+tableName
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	201 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	202 queryStatement += ' WHERE trajectory_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	203 elif objectNumbers is not None:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	204 queryStatement += ' WHERE trajectory_id '+objectCriteria
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	205 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	206 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	207 queryStatement += ' ORDER BY trajectory_id, frame_number'
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	208 elif trajectoryType == 'object':
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	209 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	210 if objectNumbers is not None:
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	211 queryStatement += ' AND OF.object_id '+objectCriteria
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	212 if timeStep is not None:
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	213 queryStatement += ' AND P.frame_number%{} = 0'.format(timeStep)
712 21aeadcfbabb added script to test SQLite performance and impact of indices Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 711 diff changeset	214 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	215 elif trajectoryType in ['bbtop', 'bbbottom']:
587 cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	216 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	217 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	218 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 586 diff changeset	219 corner = 'bottom_right'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	220 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	221 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	222 queryStatement += ' WHERE object_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	223 elif objectNumbers is not None:
712 21aeadcfbabb added script to test SQLite performance and impact of indices Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 711 diff changeset	224 queryStatement += ' WHERE object_id '+objectCriteria
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	225 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	226 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	227 queryStatement += ' ORDER BY object_id, frame_number'
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	228 else:
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	229 print('Unknown trajectory type {}'.format(trajectoryType))
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	230 if queryStatement is not None:
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	231 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	232 logging.debug(queryStatement)
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	233 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	234 printDBError(error)
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	235 return []
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	236
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	237 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	238 obj = None
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	239 objects = []
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	240 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	241 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	242 objId = row[0]
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	243 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	244 objects.append(obj)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	245 elif obj is not None:
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	246 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	247 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	248 else:
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	249 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	250 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	251
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	252 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	253 objects.append(obj)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	254 elif obj is not None:
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	255 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	256
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	257 return objects
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	258
850 c724a51d4f5f modified function signature Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 849 diff changeset	259 def loadUserTypesFromTable(cursor, objectNumbers):
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	260 objectCriteria = getObjectCriteria(objectNumbers)
910 b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	261 queryStatement = 'SELECT object_id, road_user_type FROM objects'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	262 if objectNumbers is not None:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	263 queryStatement += ' WHERE object_id '+objectCriteria
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	264 cursor.execute(queryStatement)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	265 userTypes = {}
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	266 for row in cursor:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	267 userTypes[row[0]] = row[1]
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	268 return userTypes
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	269
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	270 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False, timeStep = None, tablePrefix = None):
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	271 '''Loads the trajectories (in the general sense,
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	272 either features, objects (feature groups) or bounding box series)
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	273 The number loaded is either the first objectNumbers objects,
ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	274 or the indices in objectNumbers from the database'''
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	275 connection = sqlite3.connect(filename)
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	276
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	277 if tablePrefix is None:
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	278 prefix = ''
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	279 else:
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	280 prefix = tablePrefix + '_'
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	281 objects = loadTrajectoriesFromTable(connection, prefix+'positions', trajectoryType, objectNumbers, timeStep)
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	282 objectVelocities = loadTrajectoriesFromTable(connection, prefix+'velocities', trajectoryType, objectNumbers, timeStep)
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	283
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	284 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	285 for o,v in zip(objects, objectVelocities):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	286 if o.getNum() == v.getNum():
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	287 o.velocities = v.positions
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	288 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	289 else:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	290 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
239 93c26e45efd8 modified functions to read velocities from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 235 diff changeset	291
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	292 if trajectoryType == 'object':
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	293 cursor = connection.cursor()
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	294 try:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	295 # attribute feature numbers to objects
852 45a53542e046 updated unnecessary complicated query Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 850 diff changeset	296 queryStatement = 'SELECT trajectory_id, object_id FROM objects_features'
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	297 if objectNumbers is not None:
910 b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	298 queryStatement += ' WHERE object_id '+getObjectCriteria(objectNumbers)
852 45a53542e046 updated unnecessary complicated query Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 850 diff changeset	299 queryStatement += ' ORDER BY object_id' # order is important to group all features per object
910 b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	300 logging.debug(queryStatement)
417 a2ff03a52b73 added basic logging capability for debugging Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 394 diff changeset	301 cursor.execute(queryStatement)
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	302
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	303 featureNumbers = {}
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	304 for row in cursor:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	305 objId = row[1]
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	306 if objId not in featureNumbers:
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	307 featureNumbers[objId] = [row[0]]
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	308 else:
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	309 featureNumbers[objId].append(row[0])
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	310
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	311 for obj in objects:
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	312 obj.featureNumbers = featureNumbers[obj.getNum()]
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	313
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	314 # load userType
850 c724a51d4f5f modified function signature Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 849 diff changeset	315 userTypes = loadUserTypesFromTable(cursor, objectNumbers)
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	316 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	317 obj.userType = userTypes[obj.getNum()]
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	318
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	319 if withFeatures:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	320 nFeatures = 0
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	321 for obj in objects:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	322 nFeatures = max(nFeatures, max(obj.featureNumbers))
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	323 features = loadTrajectoriesFromSqlite(filename, 'feature', nFeatures+1, timeStep = timeStep)
685 94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	324 for obj in objects:
94b291a5f933 several updates for display Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 680 diff changeset	325 obj.setFeatures(features)
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	326
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	327 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	328 printDBError(error)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	329 objects = []
263 c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 259 diff changeset	330
143 436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	331 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 78 diff changeset	332 return objects
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	333
910 b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	334 def loadObjectFeatureFrameNumbers(filename, objectNumbers = None):
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	335 'Loads the feature frame numbers for each object'
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	336 connection = sqlite3.connect(filename)
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	337 cursor = connection.cursor()
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	338 try:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	339 queryStatement = 'SELECT OF.object_id, TL.trajectory_id, TL.length FROM (SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions GROUP BY trajectory_id) TL, objects_features OF WHERE TL.trajectory_id = OF.trajectory_id'
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	340 if objectNumbers is not None:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	341 queryStatement += ' AND object_id '+getObjectCriteria(objectNumbers)
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	342 queryStatement += ' ORDER BY OF.object_id, TL.length DESC'
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	343 logging.debug(queryStatement)
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	344 cursor.execute(queryStatement)
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	345 objectFeatureNumbers = {}
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	346 for row in cursor:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	347 objId = row[0]
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	348 if objId in objectFeatureNumbers:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	349 objectFeatureNumbers[objId].append(row[1])
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	350 else:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	351 objectFeatureNumbers[objId] = [row[1]]
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	352 return objectFeatureNumbers
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	353 except sqlite3.OperationalError as error:
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	354 printDBError(error)
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	355 return None
b58a1061a717 loading is faster for longest object features Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 909 diff changeset	356
780 1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 778 diff changeset	357 def addCurvilinearTrajectoriesFromSqlite(filename, objects):
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 778 diff changeset	358 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane)
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 778 diff changeset	359 from a database to an existing MovingObject dict (indexed by each objects's num)'''
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 778 diff changeset	360 connection = sqlite3.connect(filename)
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	361 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	362
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	363 try:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	364 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number')
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	365 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	366 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	367 return []
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	368
784 30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	369 missingObjectNumbers = []
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	370 objNum = None
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	371 for row in cursor:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	372 if objNum != row[0]:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	373 objNum = row[0]
784 30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	374 if objNum in objects:
783 f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 780 diff changeset	375 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 780 diff changeset	376 else:
784 30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	377 missingObjectNumbers.append(objNum)
30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	378 if objNum in objects:
30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	379 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4])
30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	380 if len(missingObjectNumbers) > 0:
30bd0f2223b7 tweaking curvilinear trajectory loading Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 783 diff changeset	381 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers))
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	382
918 3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	383 def saveTrajectoriesToTable(connection, objects, trajectoryType, tablePrefix = None):
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	384 'Saves trajectories in table tableName'
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	385 cursor = connection.cursor()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	386 # Parse feature and/or object structure and commit to DB
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	387 if(trajectoryType == 'feature' or trajectoryType == 'object'):
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	388 # Extract features from objects
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	389 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	390 features = []
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	391 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	392 if obj.hasFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	393 features += obj.getFeatures()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	394 if len(features) == 0:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	395 print('Warning, objects have no features') # todo save centroid trajectories?
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	396 elif trajectoryType == 'feature':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	397 features = objects
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	398 # Setup feature queries
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	399 if tablePrefix is None:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	400 prefix = ''
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	401 else:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	402 prefix = tablePrefix+'_'
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	403 createTrajectoryTable(cursor, prefix+"positions")
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	404 createTrajectoryTable(cursor, prefix+"velocities")
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	405 positionQuery = insertTrajectoryQuery(prefix+"positions")
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	406 velocityQuery = insertTrajectoryQuery(prefix+"velocities")
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	407 # Setup object queries
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	408 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	409 createObjectsTable(cursor)
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	410 createObjectsFeaturesTable(cursor)
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	411 objectQuery = insertObjectQuery()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	412 objectFeatureQuery = insertObjectFeatureQuery()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	413 for feature in features:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	414 num = feature.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	415 frameNum = feature.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	416 for p in feature.getPositions():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	417 cursor.execute(positionQuery, (num, frameNum, p.x, p.y))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	418 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	419 velocities = feature.getVelocities()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	420 if velocities is not None:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	421 frameNum = feature.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	422 for v in velocities[:-1]:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	423 cursor.execute(velocityQuery, (num, frameNum, v.x, v.y))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	424 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	425 if trajectoryType == 'object':
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	426 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	427 if obj.hasFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	428 for feature in obj.getFeatures():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	429 featureNum = feature.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	430 cursor.execute(objectFeatureQuery, (obj.getNum(), featureNum))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	431 cursor.execute(objectQuery, (obj.getNum(), obj.getUserType(), 1))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	432 # Parse curvilinear position structure
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	433 elif(trajectoryType == 'curvilinear'):
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	434 createCurvilinearTrajectoryTable(cursor)
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	435 curvilinearQuery = "insert into curvilinear_positions (trajectory_id, frame_number, s_coordinate, y_coordinate, lane) values (?,?,?,?,?)"
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	436 for obj in objects:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	437 num = obj.getNum()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	438 frameNum = obj.getFirstInstant()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	439 for p in obj.getCurvilinearPositions():
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	440 cursor.execute(curvilinearQuery, (num, frameNum, p[0], p[1], p[2]))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	441 frameNum += 1
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	442 else:
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	443 print('Unknown trajectory type {}'.format(trajectoryType))
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	444 connection.commit()
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	445
3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	446 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType):
829 0ddcc41663f5 renaming Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 813 diff changeset	447 '''Writes features, ie the trajectory positions (and velocities if exist)
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	448 with their instants to a specified sqlite file
780 1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 778 diff changeset	449 Either feature positions (and velocities if they exist)
879 f9ea5083588e Initial commit of object DB storage update pstaub parents: 872 diff changeset	450 or curvilinear positions will be saved at a time'''
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	451
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	452 connection = sqlite3.connect(outputFilename)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	453 try:
918 3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	454 saveTrajectoriesToTable(connection, objects, trajectoryType, None)
778 bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	455 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	456 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	457 connection.close()
bd684e57c431 integrated code from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 777 diff changeset	458
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	459 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None, timeStep = None):
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	460 '''Loads bounding box moving object from an SQLite
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	461 (format of SQLite output by the ground truth annotation tool
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	462 or Urban Tracker
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	463
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	464 Load descriptions?'''
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	465 connection = sqlite3.connect(filename)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	466 objects = []
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	467
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	468 if objectType == 'bb':
834 119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	469 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers, timeStep)
119c4efe6398 added option to load subsampled trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 831 diff changeset	470 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers, timeStep)
850 c724a51d4f5f modified function signature Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 849 diff changeset	471 userTypes = loadUserTypesFromTable(connection.cursor(), objectNumbers) # string format is same as object
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	472
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	473 for t, b in zip(topCorners, bottomCorners):
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	474 num = t.getNum()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	475 if t.getNum() == b.getNum():
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	476 annotation = moving.BBMovingObject(num, t.getTimeInterval(), t, b, userTypes[num])
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	477 objects.append(annotation)
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	478 else:
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	479 print ('Unknown type of bounding box {}'.format(objectType))
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	480
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	481 connection.close()
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	482 return objects
588 c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 587 diff changeset	483
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	484 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	485 roadUserNumbers = list(interaction.getRoadUserNumbers())
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	486 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	487
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	488 def saveInteractions(filename, interactions):
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	489 'Saves the interactions in the table'
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	490 connection = sqlite3.connect(filename)
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	491 cursor = connection.cursor()
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	492 try:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	493 createInteractionTable(cursor)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	494 for inter in interactions:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	495 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	496 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	497 printDBError(error)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	498 connection.commit()
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	499 connection.close()
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	500
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	501 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	502 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	503 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	504 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	505
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	506 def saveIndicators(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	507 'Saves the indicator values in the table'
1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	508 connection = sqlite3.connect(filename)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	509 cursor = connection.cursor()
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	510 try:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	511 createInteractionTable(cursor)
830 2a5856961933 first working version of feature merging (works with feature grouping) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 829 diff changeset	512 createIndicatorTable(cursor)
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	513 for inter in interactions:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	514 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	515 for indicatorName in indicatorNames:
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	516 indicator = inter.getIndicator(indicatorName)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	517 if indicator is not None:
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	518 saveIndicator(cursor, inter.getNum(), indicator)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	519 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	520 printDBError(error)
341 2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	521 connection.commit()
2f39c4ed0b62 first version of indicator saving to sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 340 diff changeset	522 connection.close()
340 1046b7346886 work in progress on storing indicator values Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 335 diff changeset	523
482 f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	524 def loadInteractions(filename):
f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	525 '''Loads interaction and their indicators
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	526
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	527 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	528 interactions = []
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	529 connection = sqlite3.connect(filename)
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	530 cursor = connection.cursor()
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	531 try:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	532 cursor.execute('select INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	533 interactionNum = -1
4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	534 indicatorTypeNum = -1
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	535 tmpIndicators = {}
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	536 for row in cursor:
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	537 if row[0] != interactionNum:
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	538 interactionNum = row[0]
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	539 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	540 interactions[-1].indicators = {}
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	541 if indicatorTypeNum != row[5] or row[0] != interactionNum:
695 957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 693 diff changeset	542 indicatorTypeNum = row[5]
628 977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	543 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 619 diff changeset	544 indicatorValues = {row[6]:row[7]}
695 957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 693 diff changeset	545 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators)
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	546 else:
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	547 indicatorValues[row[6]] = row[7]
693 5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 690 diff changeset	548 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	549 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	550 printDBError(error)
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	551 return []
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	552 connection.close()
343 74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	553 return interactions
74e437ab5f11 first version of indicator loading code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 342 diff changeset	554 # load first and last object instants
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	555 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342 4d69486869a5 work on loading indicators Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 341 diff changeset	556
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	557 def createBoundingBoxTable(filename, invHomography = None):
482 f6415f012640 adding functionalities (save images directly to display trajectories to create movies Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 417 diff changeset	558 '''Create the table to store the object bounding boxes in image space
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	559 '''
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	560 connection = sqlite3.connect(filename)
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	561 cursor = connection.cursor()
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	562 try:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	563 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	564 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	565 '(SELECT object_id, frame_number, (x{}+y{}+{})/w as x, (x{}+y{}+{})/w as y from '
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	566 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate{}+P.y_coordinate{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	567 'GROUP BY object_id, frame_number')
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	568 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	569 printDBError(error)
390 12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	570 connection.commit()
12be4a0cb9aa sql code to create bounding boxes in image space Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 382 diff changeset	571 connection.close()
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	572
586 ff4f0ce46ca6 modified name for loading bounding boxes (only for display) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 585 diff changeset	573 def loadBoundingBoxTableForDisplay(filename):
647 458890c0437c cleaned functions to lead bounding boxes (for display) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 646 diff changeset	574 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	575 connection = sqlite3.connect(filename)
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	576 cursor = connection.cursor()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	577 boundingBoxes = {} # list of bounding boxes for each instant
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	578 try:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	579 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
768 f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 759 diff changeset	580 result = cursor.fetchall()
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	581 if len(result) > 0:
394 6567fee37c16 renamed table for bounding boxes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 393 diff changeset	582 cursor.execute('SELECT * FROM bounding_boxes')
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	583 for row in cursor:
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	584 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	585 except sqlite3.OperationalError as error:
491 343cfd185ca6 minor changes and reaarrangements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 482 diff changeset	586 printDBError(error)
393 eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	587 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	588 connection.close()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	589 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 390 diff changeset	590
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	591 #########################
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	592 # saving and loading for scene interpretation: POIs and Prototypes
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	593 #########################
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	594
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	595 def savePrototypesToSqlite(filename, prototypeIndices, trajectoryType, objects = None, nMatchings = None, dbFilenames = None):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	596 '''save the prototype indices
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	597 if objects is not None, the trajectories are also saved in prototype_positions and _velocities
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	598 (prototypeIndices have to be in objects
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	599 objects will be saved as features, with the centroid trajectory as if it is a feature)
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	600 nMatchings, if not None, is a list of the number of matches
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	601 dbFilenames, if not None, is a list of the DB filenames
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	602
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	603 The order of prototypeIndices, objects, nMatchings and dbFilenames should be consistent'''
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	604 connection = sqlite3.connect(filename)
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	605 cursor = connection.cursor()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	606 try:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	607 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (id INTEGER, dbfilename VARCHAR, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), nmatchings INTEGER, positions_id INTEGER, PRIMARY KEY (id, dbfilename))')
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	608 for i, protoId in enumerate(prototypeIndices):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	609 if nMatchings is not None:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	610 n = nMatchings[i]
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	611 else:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	612 n = 'NULL'
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	613 if dbFilenames is not None:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	614 dbfn = dbFilenames[i]
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	615 else:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	616 dbfn = filename
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	617 cursor.execute('INSERT INTO prototypes (id, dbfilename, trajectory_type, nmatchings, positions_id) VALUES ({},\"{}\",\"{}\",{}, {})'.format(protoId, dbfn, trajectoryType, n, i))
918 3a06007a4bb7 modularized save trajectories, added slice to Trajectory, etc Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 917 diff changeset	618 if objects is not None: # save positions and velocities
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	619 features = []
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	620 for i, o in enumerate(objects):
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	621 f = copy(o)
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	622 f.num = i
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	623 features.append(f)
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	624 saveTrajectoriesToTable(connection, features, 'feature', 'prototype')
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	625 except sqlite3.OperationalError as error:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	626 printDBError(error)
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	627 connection.commit()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	628 connection.close()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	629
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	630 def savePrototypeAssignments(filename, objects):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	631 pass
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	632
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	633 def loadPrototypesFromSqlite(filename):
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	634 'Loads prototype ids and matchings (if stored)'
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	635 connection = sqlite3.connect(filename)
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	636 cursor = connection.cursor()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	637 prototypeIndices = []
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	638 dbFilenames = []
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	639 trajectoryTypes = []
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	640 nMatchings = []
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	641 trajectoryNumbers = []
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	642 try:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	643 cursor.execute('SELECT * FROM prototypes')
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	644 for row in cursor:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	645 prototypeIndices.append(row[0])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	646 dbFilenames.append(row[1])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	647 trajectoryTypes.append(row[2])
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	648 if row[3] is not None:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	649 nMatchings.append(row[3])
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	650 if row[4] is not None:
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	651 trajectoryNumbers.append(row[4])
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	652 if tableExists(connection, 'prototype_positions'): # load prototypes trajectories
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	653 objects = loadTrajectoriesFromSqlite(filename, 'feature', trajectoryNumbers, tablePrefix = 'prototype')
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	654 else:
7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	655 objects = None
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	656 except sqlite3.OperationalError as error:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	657 printDBError(error)
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	658 connection.close()
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	659 if len(set(trajectoryTypes)) > 1:
89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	660 print('Different types of prototypes in database ({}).'.format(set(trajectoryTypes)))
919 7b3f2e0a2652 saving and loading prototype trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 918 diff changeset	661 return prototypeIndices, dbFilenames, trajectoryTypes, nMatchings, objects
917 89cc05867c4c reorg and work in progress Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 915 diff changeset	662
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	663 def savePOIs(filename, gmm, gmmType, gmmId):
871 6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	664 '''Saves a Gaussian mixture model (of class sklearn.mixture.GaussianMixture)
6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	665 gmmType is a type of GaussianMixture, learnt either from beginnings or ends of trajectories'''
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	666 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	667 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	668 if gmmType not in ['beginning', 'end']:
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	669 print('Unknown POI type {}. Exiting'.format(gmmType))
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	670 import sys
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	671 sys.exit()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	672 try:
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	673 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (poi_id INTEGER, id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covariance VARCHAR, covariance_type VARCHAR, weight, precisions_cholesky VARCHAR, PRIMARY KEY(poi_id, id))')
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	674 for i in xrange(gmm.n_components):
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	675 cursor.execute('INSERT INTO gaussians2d VALUES({}, {}, \'{}\', {}, {}, \'{}\', \'{}\', {}, \'{}\')'.format(gmmId, i, gmmType, gmm.means_[i][0], gmm.means_[i][1], str(gmm.covariances_[i].tolist()), gmm.covariance_type, gmm.weights_[i], str(gmm.precisions_cholesky_[i].tolist())))
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	676 connection.commit()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	677 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	678 printDBError(error)
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	679 connection.close()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	680
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	681 def savePOIAssignments(filename, objects):
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	682 'save the od fields of objects'
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	683 connection = sqlite3.connect(filename)
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	684 cursor = connection.cursor()
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	685 try:
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	686 cursor.execute('CREATE TABLE IF NOT EXISTS objects_pois (object_id INTEGER, origin_poi_id INTEGER, destination_poi_id INTEGER, PRIMARY KEY(object_id))')
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	687 for o in objects:
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	688 cursor.execute('INSERT INTO objects_pois VALUES({},{},{})'.format(o.getNum(), o.od[0], o.od[1]))
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	689 connection.commit()
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	690 except sqlite3.OperationalError as error:
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	691 printDBError(error)
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	692 connection.close()
13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	693
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	694 def loadPOIs(filename):
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	695 'Loads all 2D Gaussians in the database'
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	696 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	697 from ast import literal_eval
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	698 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	699 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	700 pois = []
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	701 try:
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	702 cursor.execute('SELECT * from gaussians2d')
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	703 gmmId = None
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	704 gmm = []
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	705 for row in cursor:
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	706 if gmmId is None or row[0] != gmmId:
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	707 if len(gmm) > 0:
871 6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	708 tmp = mixture.GaussianMixture(len(gmm), covarianceType)
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	709 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
871 6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	710 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	711 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	712 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	713 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	714 pois.append(tmp)
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	715 gaussian = {'type': row[2],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	716 'mean': row[3:5],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	717 'covar': array(literal_eval(row[5])),
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	718 'weight': row[7],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	719 'precisions': array(literal_eval(row[8]))}
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	720 gmm = [gaussian]
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	721 covarianceType = row[6]
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	722 gmmId = row[0]
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	723 else:
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	724 gmm.append({'type': row[2],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	725 'mean': row[3:5],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	726 'covar': array(literal_eval(row[5])),
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	727 'weight': row[7],
c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	728 'precisions': array(literal_eval(row[8]))})
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	729 if len(gmm) > 0:
871 6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	730 tmp = mixture.GaussianMixture(len(gmm), covarianceType)
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	731 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
871 6db83beb5350 work in progress to update gaussian mixtures Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 862 diff changeset	732 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	733 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	734 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
872 c70adaeeddf5 solved issue with latest version of scikit-learn Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 871 diff changeset	735 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
805 180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	736 pois.append(tmp)
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	737 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	738 printDBError(error)
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	739 connection.close()
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	740 return pois
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	741
180b6b0231c0 added saving/loading points of interests Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 784 diff changeset	742 #########################
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	743 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	744 #########################
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	745
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	746 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	747 ''' prototype dataset is a dictionary with keys== routes, values== prototypes Ids '''
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	748 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	749 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	750
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	751 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))')
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	752
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	753 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	754 if prototypes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	755 for i in prototypes[route]:
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	756 cursor.execute('insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)',(i,route[0],route[1],nMatching[route][i]))
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	757
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	758 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	759 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	760
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	761 def readPrototypesFromSqlite(filename):
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	762 '''
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	763 This function loads the prototype file in the database
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	764 It returns a dictionary for prototypes for each route and nMatching
915 13434f5017dd work to save trajectory assignment to origin and destinations Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 913 diff changeset	765 '''
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	766 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	767 nMatching={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	768
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	769 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	770 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	771
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	772 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	773 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	774 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	775 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	776 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	777
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	778 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	779 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	780 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	781 prototypes[route]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	782 prototypes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	783 nMatching[row[0]]=row[3]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	784
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	785 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	786 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	787
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	788 def writeLabelsToSqlite(labels, outputFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	789 """ labels is a dictionary with keys: routes, values: prototypes Ids
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	790 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	791 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	792 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	793
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	794 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	795
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	796 for route in labels.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	797 if labels[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	798 for i in labels[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	799 for j in labels[route][i]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	800 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	801
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	802 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	803 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	804
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	805 def loadLabelsFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	806 labels = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	807
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	808 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	809 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	810
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	811 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	812 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	813 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	814 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	815 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	816
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	817 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	818 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	819 p=row[3]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	820 if route not in labels.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	821 labels[route]={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	822 if p not in labels[route].keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	823 labels[route][p]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	824 labels[route][p].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	825
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	826 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	827 return labels
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	828
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	829 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	830 """ to match the format of second layer prototypes"""
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	831 connection = sqlite3.connect(outFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	832 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	833
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	834 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	835
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	836 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	837 if prototypes[route]!={}:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	838 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	839 if prototypes[route][i]!= []:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	840 for j in prototypes[route][i]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	841 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	842
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	843 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	844 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	845
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	846 def loadSpeedPrototypeFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	847 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	848 This function loads the prototypes table in the database of name <filename>.
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	849 """
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	850 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	851 nMatching={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	852 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	853 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	854
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	855 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	856 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	857 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	858 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	859 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	860
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	861 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	862 route=(row[2],row[3])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	863 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	864 prototypes[route]={}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	865 if row[1] not in prototypes[route].keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	866 prototypes[route][row[1]]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	867 prototypes[route][row[1]].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	868 nMatching[row[0]]=row[4]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	869
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	870 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	871 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	872
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	873
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	874 def writeRoutesToSqlite(Routes, outputFilename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	875 """ This function writes the activity path define by start and end IDs"""
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	876 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	877 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	878
777 ef6dd60be2e1 added function to save feature trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 773 diff changeset	879 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	880
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	881 for route in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	882 if Routes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	883 for i in Routes[route]:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	884 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	885
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	886 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	887 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	888
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	889 def loadRoutesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	890 Routes = {}
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	891
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	892 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	893 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	894
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	895 try:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	896 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	897 except sqlite3.OperationalError as error:
773 bf4a1790cfac minor bug and improvements Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 768 diff changeset	898 printDBError(error)
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	899 return []
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	900
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	901 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	902 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	903 if route not in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	904 Routes[route]=[]
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	905 Routes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	906
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	907 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	908 return Routes
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	909
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	910 def setRoutes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	911 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	912 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	913 for obj in objects:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	914 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum()))
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	915 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum()))
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	916 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	917 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	918
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	919 def setRoadUserTypes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	920 '''Saves the user types of the objects in the sqlite database stored in filename
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	921 The objects should exist in the objects table'''
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	922 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	923 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	924 for obj in objects:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	925 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum()))
736 967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	926 connection.commit()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	927 connection.close()
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	928
967d244968a4 work in progress on saving/loading prototypes Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 722 diff changeset	929 #########################
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	930 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	931 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	932
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	933 def openCheck(filename, option = 'r', quitting = False):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	934 '''Open file filename in read mode by default
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	935 and checks it is open'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	936 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	937 return open(filename, option)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	938 except IOError:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	939 print 'File %s could not be opened.' % filename
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	940 if quitting:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	941 from sys import exit
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	942 exit()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	943 return None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	944
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	945 def readline(f, commentCharacters = commentChar):
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	946 '''Modified readline function to skip comments
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	947 Can take a list of characters or a string (in will work in both)'''
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	948 s = f.readline()
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	949 while (len(s) > 0) and s[0] in commentCharacters:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	950 s = f.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	951 return s.strip()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	952
722 49e99ca34a7d corrected bugs in old ubc code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 718 diff changeset	953 def getLines(f, delimiterChar = delimiterChar, commentCharacters = commentChar):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	954 '''Gets a complete entry (all the lines) in between delimiterChar.'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	955 dataStrings = []
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	956 s = readline(f, commentCharacters)
722 49e99ca34a7d corrected bugs in old ubc code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 718 diff changeset	957 while len(s) > 0 and s[0] != delimiterChar:
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	958 dataStrings += [s.strip()]
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	959 s = readline(f, commentCharacters)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	960 return dataStrings
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	961
829 0ddcc41663f5 renaming Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 813 diff changeset	962 def saveList(filename, l):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	963 f = openCheck(filename, 'w')
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	964 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	965 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	966 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	967
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	968 def loadListStrings(filename, commentCharacters = commentChar):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	969 f = openCheck(filename, 'r')
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	970 result = getLines(f, commentCharacters)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	971 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	972 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	973
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	974 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = commentChar):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	975 values = []
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	976 for l in loadListStrings(filename, commentCharacters):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	977 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	978 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	979 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	980
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	981 class FakeSecHead(object):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	982 '''Add fake section header [asection]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	983
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	984 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	985 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	986 '''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	987 def __init__(self, fp):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	988 self.fp = fp
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	989 self.sechead = '[main]\n'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	990
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	991 def readline(self):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	992 if self.sechead:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	993 try: return self.sechead
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	994 finally: self.sechead = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	995 else: return self.fp.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	996
892 f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	997 def loadPemsTraffic(filename):
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	998 '''Loads traffic data downloaded from the http://pems.dot.ca.gov clearinghouse
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	999 into pandas dataframe'''
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1000 f = openCheck(filename)
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1001 l = f.readline().strip()
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1002 items = l.split(',')
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1003 headers = ['time', 'station', 'district', 'route', 'direction', 'lanetype', 'length', 'nsamples', 'pctobserved', 'flow', 'occupancy', 'speed', 'delay35', 'delay40', 'delay45', 'delay50', 'delay55', 'delay60']
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1004 nLanes = (len(items)-len(headers))/3
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1005 for i in xrange(nLanes):
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1006 headers += ['flow{}'.format(i+1), 'occupancy{}'.format(i+1), 'speed{}'.format(i+1)]
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1007 f.close()
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1008 return read_csv(filename, delimiter = ',', names = headers)
f766fe0995f4 added function for PeMS data Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 882 diff changeset	1009
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1010 def generatePDLaneColumn(data):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1011 data['LANE'] = data['LANE\LINK\NO'].astype(str)+'_'+data['LANE\INDEX'].astype(str)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1012
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1013 def convertTrajectoriesVissimToSqlite(filename):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1014 '''Relies on a system call to sqlite3
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1015 sqlite3 [file.sqlite] < import_fzp.sql'''
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1016 sqlScriptFilename = "import_fzp.sql"
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1017 # create sql file
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1018 out = openCheck(sqlScriptFilename, "w")
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1019 out.write(".separator \";\"\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1020 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1021 ".import "+filename+" curvilinear_positions\n"+
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1022 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n")
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1023 out.close()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1024 # system call
753 3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	1025 from subprocess import check_call
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	1026 out = openCheck("err.log", "w")
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	1027 check_call("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out, shell = True)
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	1028 out.close()
3d48e34db846 switched to subprocess.check_call Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 752 diff changeset	1029 shutil.os.remove(sqlScriptFilename)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1030
754 782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1031 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds):
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1032 '''Finds the ids of the objects that go through any of the link in the list linkIds'''
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1033 connection = sqlite3.connect(filename)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1034 cursor = connection.cursor()
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1035 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')'
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1036 try:
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1037 cursor.execute(queryStatement)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1038 return [row[0] for row in cursor]
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1039 except sqlite3.OperationalError as error:
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1040 printDBError(error)
782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1041
759 a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1042 def getNObjectsInLinkFromVissimFile(filename, linkIds):
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1043 '''Returns the number of objects that traveled through the link ids'''
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1044 connection = sqlite3.connect(filename)
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1045 cursor = connection.cursor()
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1046 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id'
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1047 try:
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1048 cursor.execute(queryStatement)
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1049 return {row[0]:row[1] for row in cursor}
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1050 except sqlite3.OperationalError as error:
a05b70f307dd added function to count vehicles per VISSIM link Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 756 diff changeset	1051 printDBError(error)
754 782e8fd3672c added function to find object ids going through some vissim links Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 753 diff changeset	1052
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1053 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	1054 '''Reads data from VISSIM .fzp trajectory file
755 f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	1055 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second)
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	1056 for example, there seems to be 10 simulation steps per simulated second in VISSIM,
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 754 diff changeset	1057 so simulationStepsPerTimeUnit should be 10,
527 37830a831818 loading from VISSIM trajectory data works Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 526 diff changeset	1058 so that all times correspond to the number of the simulation step (and can be stored as integers)
660 994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	1059
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	1060 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	1061 (if the object has no such position, it won't be loaded)
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	1062
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1063 Assumed to be sorted over time
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1064 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id'''
525 7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	1065 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 524 diff changeset	1066
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1067 if usePandas:
718 2cd245cb780d added option to set low_memory = False for pandas.read_csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 714 diff changeset	1068 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1069 generatePDLaneColumn(data)
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1070 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660 994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	1071 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 649 diff changeset	1072 data = data[data['TIME']>=warmUpLastInstant]
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1073 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	1074 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642 932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1075 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1076 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1077 tmp = data[data['NO'] == objNum]
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1078 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']))
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 636 diff changeset	1079 # positions should be rounded to nDecimals decimals only
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	1080 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1081 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers:
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1082 objects.values()
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1083 else:
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1084 if filename.endswith(".fzp"):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1085 inputfile = openCheck(filename, quitting = True)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1086 line = readline(inputfile, '*$')
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1087 while len(line) > 0:#for line in inputfile:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1088 data = line.strip().split(';')
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1089 objNum = int(data[1])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1090 instant = float(data[0])*simulationStepsPerTimeUnit
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1091 s = float(data[4])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1092 y = float(data[5])
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1093 lane = data[2]+'_'+data[3]
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1094 if objNum not in objects:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1095 if warmUpLastInstant is None or instant >= warmUpLastInstant:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1096 if objectNumbers is None or len(objects) < objectNumbers:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1097 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1098 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1099 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1100 objects[objNum].timeInterval.last = instant
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1101 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1102 line = readline(inputfile, '*$')
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1103 elif filename.endswith(".sqlite"):
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1104 connection = sqlite3.connect(filename)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1105 cursor = connection.cursor()
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1106 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions'
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1107 if objectNumbers is not None:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1108 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers)
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1109 queryStatement += ' ORDER BY trajectory_id, t'
752 14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1110 try:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1111 cursor.execute(queryStatement)
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1112 for row in cursor:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1113 objNum = row[1]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1114 instant = row[0]*simulationStepsPerTimeUnit
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1115 s = row[4]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1116 y = row[5]
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1117 lane = '{}_{}'.format(row[2], row[3])
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1118 if objNum not in objects:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1119 if warmUpLastInstant is None or instant >= warmUpLastInstant:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1120 if objectNumbers is None or len(objects) < objectNumbers:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1121 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1122 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1123 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1124 objects[objNum].timeInterval.last = instant
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1125 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1126 except sqlite3.OperationalError as error:
14963a9c3b09 debug finished Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 751 diff changeset	1127 printDBError(error)
750 6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1128 else:
6049e9b6902c work in progress storage vissim sqlite Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 744 diff changeset	1129 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
751 79405a938407 corrected bug Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 750 diff changeset	1130 return objects.values()
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1131
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1132 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1133 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1134
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1135 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1136 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1137 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1138 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1139 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1140 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1141 indices = indices \| (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1142 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1143 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1144 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1145
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1146 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1147 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1148 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1149
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1150 Vehicles are considered finally stationary
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1151 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1152 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1153 (format as string x_y where x is link index and y is lane index)'''
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1154 if filename.endswith(".fzp"):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1155 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1156 if lanes is not None:
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1157 columns += ['LANE\LINK\NO', 'LANE\INDEX']
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1158 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1159 data = selectPDLanes(data, lanes)
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1160 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1161
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1162 nStationary = 0
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1163 nVehicles = 0
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1164 for name, group in data.groupby(['NO'], sort = False):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1165 nVehicles += 1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1166 positions = array(group['POS'])
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1167 diff = positions[1:]-positions[:-1]
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1168 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1169 nStationary += 1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1170 elif filename.endswith(".sqlite"):
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1171 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1172 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time?
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1173 pass
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1174 else:
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1175 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
646 6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1176
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1177 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 645 diff changeset	1178
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	1179 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True):
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1180 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1181
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1182 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1183 one checks when the sign of the position difference inverts
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1184 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1185 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1186 (format as string x_y where x is link index and y is lane index)'''
718 2cd245cb780d added option to set low_memory = False for pandas.read_csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 714 diff changeset	1187 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649 df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 647 diff changeset	1188 data = selectPDLanes(data, lanes)
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	1189 data = data.convert_objects(convert_numeric=True)
ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	1190
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1191 merged = merge(data, data, how='inner', left_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], sort = False)
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1192 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1193
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1194 nCollisions = 0
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1195 for name, group in merged.groupby(['LANE\LINK\NO', 'LANE\INDEX', 'NO_x', 'NO_y']):
744 ed6ff2ec0aeb bug correction from Laurent Gauthier Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 740 diff changeset	1196 diff = group['POS_x']-group['POS_y']
704 f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	1197 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 702 diff changeset	1198 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1199 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1200 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1201 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1202 nCollisions += 1
756 a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1203
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1204 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by??
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1205 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id;
a73f43aac00e moved pandas imports, sql in comments Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 755 diff changeset	1206 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0
645 5ed2118c959d added function to count collisions in vissim Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 643 diff changeset	1207 return nCollisions
524 1dced8932b08 corrected bugs Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 514 diff changeset	1208
173 319a04ba9033 function name change Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 143 diff changeset	1209 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1210 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1211 and returns the list of Feature objects'''
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1212 objects = []
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1213
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1214 inputfile = openCheck(filename, quitting = True)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1215
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1216 def createObject(numbers):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1217 firstFrameNum = int(numbers[1])
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1218 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1219
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1220 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1221 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1222 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1223 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1224 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1225 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1226 userType = int(numbers[10]))
78 99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1227 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1228 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1229 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1230 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1231 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1232 obj.timeHeadways = [float(numbers[17])] # seconds
327 42f2b46ec210 added class for trajectories in curvilinear coordinates Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 326 diff changeset	1233 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1234 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1235 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1236 return obj
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1237
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1238 numbers = readline(inputfile).strip().split()
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1239 if (len(numbers) > 0):
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1240 obj = createObject(numbers)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1241
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1242 for line in inputfile:
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1243 numbers = line.strip().split()
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1244 if obj.getNum() != int(numbers[0]):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1245 # check and adapt the length to deal with issues in NGSIM data
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1246 if (obj.length() != obj.positions.length()):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1247 print 'length pb with object %s (%d,%d)' % (obj.getNum(),obj.length(),obj.positions.length())
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1248 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1249 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1250 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1251 if (nObjects>0) and (len(objects)>=nObjects):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1252 break
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1253 obj = createObject(numbers)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1254 else:
327 42f2b46ec210 added class for trajectories in curvilinear coordinates Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 326 diff changeset	1255 obj.laneNums.append(int(numbers[13]))
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1256 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542 a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 541 diff changeset	1257 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1258 obj.speeds.append(float(numbers[11]))
78 99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1259 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1260 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1261 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 72 diff changeset	1262 obj.timeHeadways.append(float(numbers[17]))
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1263
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1264 if (obj.size[0] != float(numbers[8])):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1265 print 'changed length obj %d' % (obj.getNum())
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1266 if (obj.size[1] != float(numbers[9])):
329 a70c205ebdd9 added sqlite code, in particular to load and save road user type Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 327 diff changeset	1267 print 'changed width obj %d' % (obj.getNum())
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1268
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1269 inputfile.close()
7 ffddccfab7f9 loading shell objects from NGSIM works Nicolas Saunier <nico@confins.net> parents: 0 diff changeset	1270 return objects
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1271
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1272 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1273 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1274 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1275 if append:
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1276 out = openCheck(outputfile,'a')
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1277 else:
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1278 out = openCheck(outputfile,'w')
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1279 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1280
564 36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 552 diff changeset	1281 features = loadNgsimFile(inputfile, sequenceNum)
0 aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1282 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1283 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1284 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1285
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1286 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1287
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1288 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM Nicolas Saunier <nico@confins.net> parents: diff changeset	1289
895 739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1290 def loadPinholeCameraModel(filename, tanalystFormat = True):
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1291 '''Loads the data from a file containing the camera parameters
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1292 (pinhole camera model, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1293 and returns a dictionary'''
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1294 if tanalystFormat:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1295 f = openCheck(filename, quitting = True)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1296 content = getLines(f)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1297 cameraData = {}
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1298 for l in content:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1299 tmp = l.split(':')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1300 cameraData[tmp[0]] = float(tmp[1].strip().replace(',','.'))
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1301 return cameraData
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1302 else:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1303 print('Unknown camera model (not tanalyst format')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1304 return None
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 893 diff changeset	1305
829 0ddcc41663f5 renaming Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 813 diff changeset	1306 def savePositionsToCsv(f, obj):
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1307 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1308 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1309 curvilinearPositions = obj.getCurvilinearPositions()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1310 for i in xrange(int(obj.length())):
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1311 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1312 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636 3058e00887bc removed all issues because of tests with None, using is instead of == or != Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 628 diff changeset	1313 if curvilinearPositions is not None:
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1314 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1315 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1316 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1317
829 0ddcc41663f5 renaming Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 813 diff changeset	1318 def saveTrajectoriesToCsv(filename, objects):
514 1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 509 diff changeset	1319 f = openCheck(filename, 'w')
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1320 for i,obj in enumerate(objects):
829 0ddcc41663f5 renaming Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 813 diff changeset	1321 savePositionsToCsv(f, obj)
335 3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1322 f.close()
3950bfe22768 added functions to export trajectories to csv Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 330 diff changeset	1323
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1324
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1325 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1326 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1327 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1328
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1329 class ClassifierParameters(VideoFilenameAddable):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1330 'Class for the parameters of object classifiers'
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1331 def loadConfigFile(self, filename):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1332 from ConfigParser import ConfigParser
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1333
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1334 config = ConfigParser()
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1335 config.readfp(FakeSecHead(openCheck(filename)))
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1336 self.sectionHeader = config.sections()[0]
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1337
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1338 self.pedBikeCarSVMFilename = config.get(self.sectionHeader, 'pbv-svm-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1339 self.bikeCarSVMFilename = config.get(self.sectionHeader, 'bv-svm-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1340 self.percentIncreaseCrop = config.getfloat(self.sectionHeader, 'percent-increase-crop')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1341 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1342 x = config.getint(self.sectionHeader, 'hog-rescale-size')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1343 self.hogRescaleSize = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1344 self.hogNOrientations = config.getint(self.sectionHeader, 'hog-norientations')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1345 x = config.getint(self.sectionHeader, 'hog-npixels-cell')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1346 self.hogNPixelsPerCell = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1347 x = config.getint(self.sectionHeader, 'hog-ncells-block')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1348 self.hogNCellsPerBlock = (x, x)
893 ff92801e5c54 updated hog to scikit-image 0.13 (needed to add a block_norm attribute in classifier.cfg) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 892 diff changeset	1349 self.hogBlockNorm = config.get(self.sectionHeader, 'hog-block-norm')
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1350
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1351 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1352 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1353 self.speedAggregationQuantile = config.getint(self.sectionHeader, 'speed-aggregation-quantile')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1354 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
899 1466a63dd1cf added a new classification parameter Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 895 diff changeset	1355 self.maxPercentUnknown = config.getfloat(self.sectionHeader, 'max-prop-unknown-appearance')
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1356 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1357 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1358 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1359 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1360 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1361 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1362 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1363 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1364
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1365 def __init__(self, filename = None):
813 ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1366 if filename is not None and path.exists(filename):
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1367 self.loadConfigFile(filename)
813 ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1368 else:
ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1369 print('Configuration filename {} could not be loaded.'.format(filename))
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1370
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1371 def convertToFrames(self, frameRate, speedRatio = 3.6):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1372 '''Converts parameters with a relationship to time in 'native' frame time
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1373 speedRatio is the conversion from the speed unit in the config file
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1374 to the distance per second
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1375
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1376 ie param(config file) = speedRatio x fps x param(used in program)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1377 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1378 denominator = frameRate*speedRatio
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1379 #denominator2 = denominator**2
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1380 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1381 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1382 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1383 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1384 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1385 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1386 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1387 # special case for the lognormal distribution
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1388 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
854 33d296984dd8 rework and more info on speed probabilities for classification Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 852 diff changeset	1389 #self.scaleCyclistSpeed = self.scaleCyclistSpeed # no modification of scale
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1390
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1391
664 455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 660 diff changeset	1392 class ProcessParameters(VideoFilenameAddable):
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1393 '''Class for all parameters controlling data processing: input,
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1394 method parameters, etc. for tracking and safety
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1395
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1396 Note: framerate is already taken into account'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1397
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1398 def loadConfigFile(self, filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1399 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1400
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1401 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1402 config.readfp(FakeSecHead(openCheck(filename)))
861 f9c9457b60c2 modification of storage of intrinsic camera and distortion parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 854 diff changeset	1403
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1404 self.sectionHeader = config.sections()[0]
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1405 # Tracking/display parameters
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1406 self.videoFilename = config.get(self.sectionHeader, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1407 self.databaseFilename = config.get(self.sectionHeader, 'database-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1408 self.homographyFilename = config.get(self.sectionHeader, 'homography-filename')
882 4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 880 diff changeset	1409 if path.exists(self.homographyFilename):
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 880 diff changeset	1410 self.homography = loadtxt(self.homographyFilename)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1411 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1412 self.homography = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1413 self.intrinsicCameraFilename = config.get(self.sectionHeader, 'intrinsic-camera-filename')
882 4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 880 diff changeset	1414 if path.exists(self.intrinsicCameraFilename):
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ) Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 880 diff changeset	1415 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1416 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1417 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1418 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1419 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1420 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1421 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1422 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1423 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
377 2aed569f39e7 added utils Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 344 diff changeset	1424
812 21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1425 self.classifierFilename = config.get(self.sectionHeader, 'classifier-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 805 diff changeset	1426
537 6c264b914846 work on classification parameters Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 536 diff changeset	1427 # Safety parameters
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1428 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1429 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1430 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1431 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1432 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1433 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1434 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1435 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1436 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1437 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1438 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1439 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1440
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1441 def __init__(self, filename = None):
813 ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1442 if filename is not None and path.exists(filename):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1443 self.loadConfigFile(filename)
813 ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1444 else:
ef8795dba5ed avoid crash for missing configuration filename Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 812 diff changeset	1445 print('Configuration filename {} could not be loaded.'.format(filename))
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1446
901 753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1447 def processVideoArguments(args):
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1448 '''Loads information from configuration file
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1449 then checks what was passed on the command line
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1450 for override (eg video filename and database filename'''
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1451 if args.configFilename is not None: # consider there is a configuration file
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1452 params = ProcessParameters(args.configFilename)
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1453 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1454 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1455 if params.homography is not None:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1456 invHomography = linalg.inv(params.homography)
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1457 else:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1458 invHomography = None
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1459 intrinsicCameraMatrix = params.intrinsicCameraMatrix
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1460 distortionCoefficients = params.distortionCoefficients
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1461 undistortedImageMultiplication = params.undistortedImageMultiplication
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1462 undistort = params.undistort
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1463 firstFrameNum = params.firstFrameNum
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1464 else:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1465 invHomography = None
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1466 undistort = False
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1467 intrinsicCameraMatrix = None
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1468 distortionCoefficients = []
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1469 undistortedImageMultiplication = None
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1470 undistort = False
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1471 firstFrameNum = 0
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1472
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1473 # override video and database filenames if present on command line
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1474 if args.videoFilename is not None:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1475 videoFilename = args.videoFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1476 else:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1477 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1478 if args.databaseFilename is not None:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1479 databaseFilename = args.databaseFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1480 else:
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1481 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1482
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1483 return params, videoFilename, databaseFilename, invHomography, intrinsicCameraMatrix, distortionCoefficients, undistortedImageMultiplication, undistort, firstFrameNum
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1484
753a081989e2 factorized some argument handling code Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 899 diff changeset	1485 # deprecated
665 15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 664 diff changeset	1486 class SceneParameters(object):
509 935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1487 def __init__(self, config, sectionName):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1488 from ConfigParser import NoOptionError
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1489 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1490 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1491 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1492 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1493 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1494 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1495 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1496 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1497 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1498 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1499 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1500 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1501 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1502 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1503 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1504
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1505 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1506 def loadConfigFile(filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1507 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1508 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1509 config.readfp(openCheck(filename))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1510 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1511 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1512 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1513 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 491 diff changeset	1514
382 ba813f148ade development for clustering Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 377 diff changeset	1515
215 5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1516 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1517 import doctest
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1518 import unittest
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1519 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 212 diff changeset	1520 unittest.TextTestRunner().run(suite)
72 575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1521 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class Nicolas Saunier <nicolas.saunier@polymtl.ca> parents: 50 diff changeset	1522 # #doctest.testfile("example.txt")

Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 919:7b3f2e0a2652