Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 259:8ab76b95ee72

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added code to save collision points and crossing zones in txt files
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Tue, 24 Jul 2012 15:18:12 -0400
parents 583a2c4622f9
children c71540470057
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1 #! /usr/bin/env python
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
2 # -*- coding: utf-8 -*-
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
4
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
5 import utils
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
6 import moving
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
7
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
8 __metaclass__ = type
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
9
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
10
50
7b06d649122b re-arrangement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 48
diff changeset
11 ngsimUserTypes = {'twowheels':1,
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
12 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
13 'truck':3}
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
14
259
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
15 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
16 # txt files
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
17 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
18
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
19
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
20
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
21 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
22 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
23 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
24
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
25 def writeTrajectoriesToSqlite(objects, outFile, trajectoryType, objectNumbers = -1):
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
26 """
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
27 This function writers trajectories to a specified sqlite file
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
28 @param[in] objects -> a list of trajectories
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
29 @param[in] trajectoryType -
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
30 @param[out] outFile -> the .sqlite file containting the written objects
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
31 @param[in] objectNumber : number of objects loaded
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
32 """
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
33
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
34 import sqlite3
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
35 connection = sqlite3.connect(outFile)
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
36 cursor = connection.cursor()
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
37
212
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
38 schema = "CREATE TABLE \"positions\"(trajectory_id INTEGER,frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))"
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
39 cursor.execute(schema)
212
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
40
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
41 trajectory_id = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
42 frame_number = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
43 if trajectoryType == 'feature':
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
44 if type(objectNumbers) == int and objectNumbers == -1:
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
45 for trajectory in objects:
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
46 trajectory_id += 1
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
47 frame_number = 0
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
48 for position in trajectory.getPositions():
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
49 frame_number += 1
212
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
50 query = "insert into positions (trajectory_id, frame_number, x_coordinate, y_coordinate) values (?,?,?,?)"
ce44605f888a minor modifications
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 211
diff changeset
51 cursor.execute(query,(trajectory_id,frame_number,position.x,position.y))
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
52
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
53 connection.commit()
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
54 connection.close()
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
55
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
56 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
57 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
58 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
59 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
60 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
61 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
62
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
63 import sqlite3
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
64 connection = sqlite3.connect(filename)
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
65 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
66
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
67 try:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
68 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
69 except sqlite3.OperationalError as err:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
70 print('DB Error: {0}'.format(err))
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
71 return []
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
72
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
73 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
74 matched_indexes.append((row[0],row[1]))
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
75
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
76 connection.close()
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
77 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
78
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
79 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = -1):
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
80 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
81 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
82
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
83 returns a moving object'''
246
583a2c4622f9 created new module for algorithms with function to extract speeds
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 239
diff changeset
84 import sqlite3
583a2c4622f9 created new module for algorithms with function to extract speeds
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 239
diff changeset
85
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
86 cursor = connection.cursor()
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
87
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
88 try:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
89 if trajectoryType == 'feature':
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
90 if type(objectNumbers) == int:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
91 if objectNumbers == -1:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
92 cursor.execute('SELECT * from '+tableName+' order by trajectory_id, frame_number')
218
b5772df11b37 corrected bugs to load objects and display trajectories over videos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 215
diff changeset
93 else:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
94 cursor.execute('SELECT * from {0} where trajectory_id between 0 and {1} order by trajectory_id, frame_number'.format(tableName, objectNumbers))
218
b5772df11b37 corrected bugs to load objects and display trajectories over videos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 215
diff changeset
95 elif type(objectNumbers) == list:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
96 cursor.execute('SELECT * from '+tableName+' where trajectory_id in ('+', '.join([str(n) for n in objectNumbers])+') order by trajectory_id, frame_number')
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
97 elif trajectoryType == 'object':
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
98 if type(objectNumbers) == int:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
99 if objectNumbers == -1:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
100 cursor.execute('SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF where P.trajectory_id = OF.trajectory_id group by object_id, frame_number')
218
b5772df11b37 corrected bugs to load objects and display trajectories over videos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 215
diff changeset
101 else:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
102 cursor.execute('SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from {0} P, objects_features OF where P.trajectory_id = OF.trajectory_id and OF.object_id between 0 and {1} group by object_id, frame_number'.format(tableName, objectNumbers))
218
b5772df11b37 corrected bugs to load objects and display trajectories over videos
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 215
diff changeset
103 elif type(objectNumbers) == list:
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
104 cursor.execute('SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF where P.trajectory_id = OF.trajectory_id and OF.object_id in ('+', '.join([str(n) for n in objectNumbers])+') group by object_id, frame_number')
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
105 else:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
106 print('no trajectory type was chosen')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
107 except sqlite3.OperationalError as err:
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
108 print('DB Error: {0}'.format(err))
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
109 return []
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
110
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
111 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
112 obj = None
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
113 objects = []
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
114 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
115 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
116 objId = row[0]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
117 if obj:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
118 objects.append(obj)
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
119 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
120 else:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
121 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
122 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
123
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
124 if obj:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
125 objects.append(obj)
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
126
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
127 return objects
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
128
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
129 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = -1):
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
130 '''Loads nObjects or the indices in objectNumbers from the database
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
131 TODO: load feature numbers and not average feature trajectories
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
132 TODO: other ways of averaging trajectories (load all points, sorted by frame_number and leave the agregation to be done in python)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
133 '''
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
134 import sqlite3
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
135
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
136 connection = sqlite3.connect(filename) # add test if it open
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
137
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
138 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
139 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers)
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
140
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
141 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
142 for o,v in zip(objects, objectVelocities):
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
143 if o.num == v.num:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
144 o.velocities = v.positions
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
145 else:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
146 print('Could not match positions {0} with velocities {1}'.format(o.num, v.num))
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
147
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
148 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
149 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
150
235
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
151 def removeObjectsFromSqlite(filename):
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
152 'Removes the objects and object_features tables in the filename'
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
153 import sqlite3
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
154 connection = sqlite3.connect(filename)
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
155 utils.dropTables(connection, ['objects', 'objects_features'])
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
156 connection.close()
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
157
173
319a04ba9033 function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 143
diff changeset
158 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
159 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
160 and returns the list of Feature objects'''
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
161 objects = []
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
162
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
163 input = utils.openCheck(filename)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
164 if not input:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
165 import sys
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
166 sys.exit()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
167
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
168 def createObject(numbers):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
169 firstFrameNum = int(numbers[1])
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
170 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
171
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
172 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
173 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
174 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
175 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
176 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
177 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
178 userType = int(numbers[10]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
179 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
180 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
181 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
182 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
183 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
184 obj.timeHeadways = [float(numbers[17])] # seconds
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
185 obj.curvilinearPositions = moving.Trajectory([[float(numbers[5])],[float(numbers[4])]]) # X is the longitudinal coordinate
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
186 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
187 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
188 return obj
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
189
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
190 numbers = input.readline().strip().split()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
191 if (len(numbers) > 0):
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
192 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
193
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
194 for line in input:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
195 numbers = line.strip().split()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
196 if obj.num != int(numbers[0]):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
197 # check and adapt the length to deal with issues in NGSIM data
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
198 if (obj.length() != obj.positions.length()):
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
199 print 'length pb with object %s (%d,%d)' % (obj.num,obj.length(),obj.positions.length())
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
200 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
201 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
202 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
203 if (nObjects>0) and (len(objects)>=nObjects):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
204 break
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
205 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
206 else:
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
207 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
208 obj.curvilinearPositions.addPositionXY(float(numbers[5]), float(numbers[4]))
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
209 obj.speeds.append(float(numbers[11]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
210 obj.laneNums.append(int(numbers[13]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
211 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
212 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
213 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
214 obj.timeHeadways.append(float(numbers[17]))
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
215
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
216 if (obj.size[0] != float(numbers[8])):
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
217 print 'changed length obj %d' % (f.num)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
218 if (obj.size[1] != float(numbers[9])):
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
219 print 'changed width obj %d' % (f.num)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
220
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
221 input.close()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
222 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
223
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
224 def convertNgsimFile(inFile, outFile, append = False, nObjects = -1, sequenceNum = 0):
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
225 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
226 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
227 if append:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
228 out = open(outFile,'a')
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
229 else:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
230 out = open(outFile,'w')
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
231 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
232
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
233 features = loadNgsimFile(inFile, sequenceNum)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
234 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
235 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
236 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
237
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
238 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
239
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
240 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
241
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
242 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
243 import doctest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
244 import unittest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
245 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
246 unittest.TextTestRunner().run(suite)
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
247 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
248 # #doctest.testfile("example.txt")