tools.lib package
Subpackages
- tools.lib.tests package
- Submodules
- tools.lib.tests.test_caching module
- tools.lib.tests.test_comma_car_segments module
- tools.lib.tests.test_logreader module
TestLogReader
TestLogReader.test_auto_mode()
TestLogReader.test_auto_source_scenarios
TestLogReader.test_auto_source_scenarios_0()
TestLogReader.test_auto_source_scenarios_1()
TestLogReader.test_bad_ranges
TestLogReader.test_bad_ranges_0_344c5c15b34f2d8a_2024_01_03_09_37_12_()
TestLogReader.test_bad_ranges_1_344c5c15b34f2d8a_2024_01_03_09_37_12_()
TestLogReader.test_bad_ranges_2_344c5c15b34f2d8a_2024_01_03_09_37_12_4_2()
TestLogReader.test_bad_ranges_3_344c5c15b34f2d8a_2024_01_03_09_37_12_a()
TestLogReader.test_bad_ranges_4_344c5c15b34f2d8a_2024_01_03_09_37_12_j()
TestLogReader.test_bad_ranges_5_344c5c15b34f2d8a_2024_01_03_09_37_12_0_1_2_3()
TestLogReader.test_bad_ranges_6_344c5c15b34f2d8a_2024_01_03_09_37_12_3()
TestLogReader.test_bad_ranges_7_344c5c15b34f2d8a_2024_01_03_09_37_123()
TestLogReader.test_bad_ranges_8_344c5c15b34f2d8a_2024_01_03_09_37_12_3()
TestLogReader.test_bad_ranges_9_344c5c15b34f2d8a_2024_01_03_09_37_12_3a()
TestLogReader.test_canonical_name
TestLogReader.test_canonical_name_0_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_canonical_name_1_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_canonical_name_2_344c5c15b34f2d8a_2024_01_03_09_37_12_5()
TestLogReader.test_canonical_name_3_344c5c15b34f2d8a_2024_01_03_09_37_12_0_q()
TestLogReader.test_canonical_name_4_344c5c15b34f2d8a_2024_01_03_09_37_12_5_6_r()
TestLogReader.test_canonical_name_5_344c5c15b34f2d8a_2024_01_03_09_37_12_5()
TestLogReader.test_direct_parsing
TestLogReader.test_direct_parsing_0()
TestLogReader.test_direct_parsing_1()
TestLogReader.test_helpers()
TestLogReader.test_indirect_parsing
TestLogReader.test_indirect_parsing_00_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_indirect_parsing_01_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_indirect_parsing_02_344c5c15b34f2d8a_2024_01_03_09_37_12_0()
TestLogReader.test_indirect_parsing_03_344c5c15b34f2d8a_2024_01_03_09_37_12_5()
TestLogReader.test_indirect_parsing_04_344c5c15b34f2d8a_2024_01_03_09_37_12_0()
TestLogReader.test_indirect_parsing_05_344c5c15b34f2d8a_2024_01_03_09_37_12_5()
TestLogReader.test_indirect_parsing_06_344c5c15b34f2d8a_2024_01_03_09_37_12_0_10()
TestLogReader.test_indirect_parsing_07_344c5c15b34f2d8a_2024_01_03_09_37_12_0_0()
TestLogReader.test_indirect_parsing_08_344c5c15b34f2d8a_2024_01_03_09_37_12_4_6()
TestLogReader.test_indirect_parsing_09_344c5c15b34f2d8a_2024_01_03_09_37_12_0_1()
TestLogReader.test_indirect_parsing_10_344c5c15b34f2d8a_2024_01_03_09_37_12_5()
TestLogReader.test_indirect_parsing_11_344c5c15b34f2d8a_2024_01_03_09_37_12_2_()
TestLogReader.test_indirect_parsing_12_344c5c15b34f2d8a_2024_01_03_09_37_12_2_1()
TestLogReader.test_indirect_parsing_13_344c5c15b34f2d8a_2024_01_03_09_37_12_1()
TestLogReader.test_indirect_parsing_14_344c5c15b34f2d8a_2024_01_03_09_37_12_2()
TestLogReader.test_indirect_parsing_15_344c5c15b34f2d8a_2024_01_03_09_37_12_2_1()
TestLogReader.test_indirect_parsing_16_344c5c15b34f2d8a_2024_01_03_09_37_12_4_2()
TestLogReader.test_indirect_parsing_17_344c5c15b34f2d8a_2024_01_03_09_37_12_10_2()
TestLogReader.test_indirect_parsing_18_344c5c15b34f2d8a_2024_01_03_09_37_12_5_2()
TestLogReader.test_indirect_parsing_19_https_useradmin_comma_ai_onebox_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_indirect_parsing_20_https_useradmin_comma_ai_onebox_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_indirect_parsing_21_https_useradmin_comma_ai_onebox_344c5c15b34f2d8a_7C2024_01_03_09_37_12()
TestLogReader.test_indirect_parsing_22_https_cabana_comma_ai_route_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_list()
TestLogReader.test_modes()
TestLogReader.test_modes_from_name()
TestLogReader.test_multiple_iterations()
TestLogReader.test_only_union_types()
TestLogReader.test_run_across_segments
TestLogReader.test_run_across_segments_0()
TestLogReader.test_run_across_segments_1()
TestLogReader.test_slicing_api_call
TestLogReader.test_slicing_api_call_0_344c5c15b34f2d8a_2024_01_03_09_37_12_0()
TestLogReader.test_slicing_api_call_1_344c5c15b34f2d8a_2024_01_03_09_37_12_2()
TestLogReader.test_slicing_api_call_2_344c5c15b34f2d8a_2024_01_03_09_37_12_0_()
TestLogReader.test_slicing_api_call_3_344c5c15b34f2d8a_2024_01_03_09_37_12_1()
TestLogReader.test_slicing_api_call_4_344c5c15b34f2d8a_2024_01_03_09_37_12()
TestLogReader.test_sort_by_time()
noop()
setup_source_scenario()
- tools.lib.tests.test_readers module
- tools.lib.tests.test_route_library module
- Module contents
Submodules
tools.lib.api module
tools.lib.auth module
Usage:
usage: auth.py [-h] [{google,apple,github,jwt}] [jwt]
Login to your comma account
positional arguments:
{google,apple,github,jwt}
jwt
optional arguments:
-h, --help show this help message and exit
Examples:
./auth.py # Log in with google account
./auth.py github # Log in with GitHub Account
./auth.py jwt ey......hw # Log in with a JWT from https://jwt.comma.ai, for use in CI
- class tools.lib.auth.ClientRedirectHandler(request, client_address, server)[source]
Bases:
BaseHTTPRequestHandler
- log_message(*args)[source]
Log an arbitrary message.
This is used by all other logging functions. Override it if you have specific logging wishes.
The first argument, FORMAT, is a format string for the message to be logged. If the format string contains any % escapes requiring parameters, they should be specified as subsequent arguments (it’s just like printf!).
The client ip and current date/time are prefixed to every message.
Unicode control characters are replaced with escaped hex before writing the output to stderr.
tools.lib.auth_config module
tools.lib.azure_container module
tools.lib.bootlog module
tools.lib.cache module
tools.lib.comma_car_segments module
tools.lib.exceptions module
tools.lib.filereader module
tools.lib.framereader module
- tools.lib.framereader.FrameReader(fn, cache_dir='/root/.commacache', readahead=False, readbehind=False, index_data=None)[source]
- class tools.lib.framereader.FrameType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
Bases:
IntEnum
- h265_stream = 2
- raw = 1
- class tools.lib.framereader.GOPFrameReader(readahead=False, readbehind=False)[source]
Bases:
BaseFrameReader
- class tools.lib.framereader.RawFrameReader(fn)[source]
Bases:
BaseFrameReader
- class tools.lib.framereader.StreamFrameReader(fn, frame_type, index_data, readahead=False, readbehind=False)[source]
Bases:
StreamGOPReader
,GOPFrameReader
tools.lib.helpers module
- class tools.lib.helpers.RE[source]
Bases:
object
- BOOTLOG_NAME = '(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))'
- DONGLE_ID = '(?P<dongle_id>[a-f0-9]{16})'
- EXPLORER_FILE = '^(?P<segment_name>(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))(?:--|/)(?P<segment_num>[0-9]+))--(?P<file_name>[a-z]+\\.[a-z0-9]+)$'
- INDEX = '-?[0-9]+'
- LOG_ID = '(?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10})))'
- LOG_ID_V2 = '(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10})'
- OP_SEGMENT_DIR = '^(?P<segment_name>(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))(?:--|/)(?P<segment_num>[0-9]+))$'
- ROUTE_NAME = '(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))'
- SEGMENT_NAME = '(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))(?:--|/)(?P<segment_num>[0-9]+)'
- SEGMENT_RANGE = '(?P<route_name>(?P<dongle_id>[a-f0-9]{16})[|_/](?P<log_id>(?:(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})|(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10}))))(?:(--|/)(?P<slice>((?P<start>-?[0-9]+)?:?(?P<end>-?[0-9]+)?:?(?P<step>-?[0-9]+)?)))?(?:/(?P<selector>([qras])))?'
- SLICE = '(?P<start>-?[0-9]+)?:?(?P<end>-?[0-9]+)?:?(?P<step>-?[0-9]+)?'
- TIMESTAMP = '(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})'
tools.lib.kbhit module
- class tools.lib.kbhit.KBHit[source]
Bases:
object
- static getarrow() int [source]
Returns an arrow-key code after kbhit() has been called. Codes are 0 : up 1 : right 2 : down 3 : left Should not be called in the same program as getch().
- static getch() str [source]
Returns a keyboard character after kbhit() has been called. Should not be called in the same program as getarrow().
tools.lib.live_logreader module
- tools.lib.live_logreader.live_logreader(services: list[str] = ['gyroscope', 'gyroscope2', 'accelerometer', 'accelerometer2', 'magnetometer', 'lightSensor', 'temperatureSensor', 'temperatureSensor2', 'gpsNMEA', 'deviceState', 'can', 'controlsState', 'pandaStates', 'peripheralState', 'radarState', 'roadEncodeIdx', 'liveTracks', 'sendcan', 'logMessage', 'errorLogMessage', 'liveCalibration', 'liveTorqueParameters', 'androidLog', 'carState', 'carControl', 'carOutput', 'longitudinalPlan', 'procLog', 'gpsLocationExternal', 'gpsLocation', 'ubloxGnss', 'qcomGnss', 'gnssMeasurements', 'clocks', 'ubloxRaw', 'liveLocationKalman', 'liveParameters', 'cameraOdometry', 'thumbnail', 'onroadEvents', 'carParams', 'roadCameraState', 'driverCameraState', 'driverEncodeIdx', 'driverStateV2', 'driverMonitoringState', 'wideRoadEncodeIdx', 'wideRoadCameraState', 'modelV2', 'managerState', 'uploaderState', 'navInstruction', 'navRoute', 'navThumbnail', 'uiPlan', 'qRoadEncodeIdx', 'userFlag', 'microphone', 'uiDebug', 'testJoystick', 'roadEncodeData', 'driverEncodeData', 'wideRoadEncodeData', 'qRoadEncodeData', 'livestreamWideRoadEncodeIdx', 'livestreamRoadEncodeIdx', 'livestreamDriverEncodeIdx', 'livestreamWideRoadEncodeData', 'livestreamRoadEncodeData', 'livestreamDriverEncodeData', 'customReservedRawData0', 'customReservedRawData1', 'customReservedRawData2'], addr: str = '127.0.0.1') Iterable[type[_DynamicStructReader]] [source]
- tools.lib.live_logreader.raw_live_logreader(services: list[str] = ['gyroscope', 'gyroscope2', 'accelerometer', 'accelerometer2', 'magnetometer', 'lightSensor', 'temperatureSensor', 'temperatureSensor2', 'gpsNMEA', 'deviceState', 'can', 'controlsState', 'pandaStates', 'peripheralState', 'radarState', 'roadEncodeIdx', 'liveTracks', 'sendcan', 'logMessage', 'errorLogMessage', 'liveCalibration', 'liveTorqueParameters', 'androidLog', 'carState', 'carControl', 'carOutput', 'longitudinalPlan', 'procLog', 'gpsLocationExternal', 'gpsLocation', 'ubloxGnss', 'qcomGnss', 'gnssMeasurements', 'clocks', 'ubloxRaw', 'liveLocationKalman', 'liveParameters', 'cameraOdometry', 'thumbnail', 'onroadEvents', 'carParams', 'roadCameraState', 'driverCameraState', 'driverEncodeIdx', 'driverStateV2', 'driverMonitoringState', 'wideRoadEncodeIdx', 'wideRoadCameraState', 'modelV2', 'managerState', 'uploaderState', 'navInstruction', 'navRoute', 'navThumbnail', 'uiPlan', 'qRoadEncodeIdx', 'userFlag', 'microphone', 'uiDebug', 'testJoystick', 'roadEncodeData', 'driverEncodeData', 'wideRoadEncodeData', 'qRoadEncodeData', 'livestreamWideRoadEncodeIdx', 'livestreamRoadEncodeIdx', 'livestreamDriverEncodeIdx', 'livestreamWideRoadEncodeData', 'livestreamRoadEncodeData', 'livestreamDriverEncodeData', 'customReservedRawData0', 'customReservedRawData1', 'customReservedRawData2'], addr: str = '127.0.0.1') Iterable[bytes] [source]
tools.lib.logreader module
- class tools.lib.logreader.LogReader(identifier: str | list[str], default_mode: ~tools.lib.logreader.ReadMode = ReadMode.RLOG, default_source=<function auto_source>, sort_by_time=False, only_union_types=False)[source]
Bases:
object
- class tools.lib.logreader.ReadMode(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
Bases:
StrEnum
- AUTO = 'a'
- AUTO_INTERACTIVE = 'i'
- QLOG = 'q'
- RLOG = 'r'
- SANITIZED = 's'
- tools.lib.logreader.apply_strategy(mode: ~tools.lib.logreader.ReadMode, rlog_paths: list[str | None], qlog_paths: list[str | None], valid_file: ~collections.abc.Callable[[str | None], bool] = <function default_valid_file>) list[str | None] [source]
- tools.lib.logreader.auto_strategy(rlog_paths: list[str | None], qlog_paths: list[str | None], interactive: bool, valid_file: Callable[[str | None], bool]) list[str | None] [source]
- tools.lib.logreader.check_source(source: Callable[[SegmentRange, ReadMode], list[str | None]], *args) list[str | None] [source]
tools.lib.openpilotci module
tools.lib.openpilotcontainers module
tools.lib.route module
- class tools.lib.route.Route(name, data_dir=None)[source]
Bases:
object
- property name
- property segments
- class tools.lib.route.RouteName(name_str: str)[source]
Bases:
object
- property canonical_name: str
- property dongle_id: str
- property time_str: str
- class tools.lib.route.Segment(name, log_path, qlog_path, camera_path, dcamera_path, ecamera_path, qcamera_path)[source]
Bases:
object
- property name
- class tools.lib.route.SegmentName(name_str: str, allow_route_name=False)[source]
Bases:
object
- property canonical_name: str
- property data_dir: str | None
- property dongle_id: str
- property segment_num: int
- property time_str: str
- class tools.lib.route.SegmentRange(segment_range: str)[source]
Bases:
object
- property dongle_id: str
- property log_id: str
- property route_name: str
- property seg_idxs: list[int]
- property selector: str | None
- property slice: str
- property timestamp: str
- tools.lib.route.get_max_seg_number_cached(sr: SegmentRange) int [source]
tools.lib.sanitizer module
- tools.lib.sanitizer.sanitize(lr: Iterable[type[_DynamicStructReader]]) Iterable[type[_DynamicStructReader]] [source]
tools.lib.url_file module
tools.lib.vidindex module
- class tools.lib.vidindex.HevcNalUnitType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
Bases:
IntEnum
- AUD_NUT = 35
- BLA_N_LP = 18
- BLA_W_LP = 16
- BLA_W_RADL = 17
- CRA_NUT = 21
- EOB_NUT = 37
- EOS_NUT = 36
- FD_NUT = 38
- IDR_N_LP = 20
- IDR_W_RADL = 19
- PPS_NUT = 34
- PREFIX_SEI_NUT = 39
- RADL_N = 6
- RADL_R = 7
- RASL_N = 8
- RASL_R = 9
- RSV_IRAP_VCL22 = 22
- RSV_IRAP_VCL23 = 23
- RSV_NVCL41 = 41
- RSV_NVCL42 = 42
- RSV_NVCL43 = 43
- RSV_NVCL44 = 44
- RSV_NVCL45 = 45
- RSV_NVCL46 = 46
- RSV_NVCL47 = 47
- RSV_VCL24 = 24
- RSV_VCL25 = 25
- RSV_VCL26 = 26
- RSV_VCL27 = 27
- RSV_VCL28 = 28
- RSV_VCL29 = 29
- RSV_VCL30 = 30
- RSV_VCL31 = 31
- RSV_VCL_N10 = 10
- RSV_VCL_N12 = 12
- RSV_VCL_N14 = 14
- RSV_VCL_R11 = 11
- RSV_VCL_R13 = 13
- RSV_VCL_R15 = 15
- SPS_NUT = 33
- STSA_N = 4
- STSA_R = 5
- SUFFIX_SEI_NUT = 40
- TRAIL_N = 0
- TRAIL_R = 1
- TSA_N = 2
- TSA_R = 3
- UNSPEC48 = 48
- UNSPEC49 = 49
- UNSPEC50 = 50
- UNSPEC51 = 51
- UNSPEC52 = 52
- UNSPEC53 = 53
- UNSPEC54 = 54
- UNSPEC55 = 55
- UNSPEC56 = 56
- UNSPEC57 = 57
- UNSPEC58 = 58
- UNSPEC59 = 59
- UNSPEC60 = 60
- UNSPEC61 = 61
- UNSPEC62 = 62
- UNSPEC63 = 63
- VPS_NUT = 32
- tools.lib.vidindex.get_hevc_nal_unit_type(dat: bytes, nal_unit_start: int) HevcNalUnitType [source]
- tools.lib.vidindex.get_hevc_slice_type(dat: bytes, nal_unit_start: int, nal_unit_type: HevcNalUnitType) tuple[int, bool] [source]