spinn_front_end_common.interface.provenance package

Module contents

class spinn_front_end_common.interface.provenance.AbstractProvidesLocalProvenanceData[source]

Bases: object

Indicates an object that provides locally obtained provenance data.

GraphProvenanceGatherer will check all Vertices and all Edges in both the MachineGraph and if it exists the ApplicationGraph

get_local_provenance_data()[source]

Get an iterable of provenance data items.

class spinn_front_end_common.interface.provenance.AbstractProvidesProvenanceDataFromMachine[source]

Bases: object

Indicates that an object provides provenance data retrieved from the machine.

get_provenance_data_from_machine(transceiver, placement)[source]

Get an iterable of provenance data items.

Parameters:
  • transceiver (Transceiver) – the SpinnMan interface object
  • placement (Placement) – the placement of the object
class spinn_front_end_common.interface.provenance.ProvenanceReader(provenance_data_path=None)[source]

Bases: object

Provides a connection to a database containing provenance for the current run and some convenience methods for extracting provenance data from it.

By default this will wrap around the database used to store the provenance of the last run. The path is not updated so this reader is not effected by a reset or an end.

The assumption is that the database is in the current provenance format. This includes both that DDL statements used to create the database but also the underlying database structure (currently sqlite3)

Warning

This class is only a wrapper around the database file so if the file is deleted the class will no longer work.

Create a wrapper around the database.

The suggested way to call this is without the provenance_data_path parameter, allowing get_last_run_database_path() to find the correct path.

Parameters:provenance_data_path (None or str) – Path to the provenance database to wrap
cores_with_late_spikes()[source]

Gets the x, y, p and count of the cores where late spikes arrived.

Cores that received spikes but where none were late are not included.

Returns:A list hopefully empty of tuples (x, y, p, count) of cores where their where late arriving spikes.
Return type:list(tuple(int, int, int, int))
get_category_timer_sum(category)[source]

Get the total runtime for one category of algorithms

Parameters:category (str) –
Returns:total off all runtimes with this category
Return type:int
get_cores_with_provenace()[source]

Gets the cores with provenance

Returns:A list tuples (x, y, p)
Return type:list(tuple(int, int, int))
get_database_handle(read_only=True, use_sqlite_rows=False)[source]

Gets a handle to the open database.

You should use this as a Python context handler. A typical usage pattern is this:

with reader.get_database_handler() as db:
    with db.transaction() as cursor:
        for row in cursor.execute(...):
            # process row

Note

This method is mainly provided as a support method for the later methods that return specific data. For new IntergationTests please add a specific method rather than call this directly.

Warning

It is the callers responsibility to close the database. The recommended usage is therefore a with statement

Parameters:
  • read_only (bool) – If true will return a readonly database
  • use_sqlite_rows (bool) – If True the results of run_query() will be Rows. If False the results of run_query() will be tuples.
Returns:

an open sqlite3 connection

Return type:

SQLiteDB

static get_last_run_database_path()[source]

Get the path of the current provenance database of the last run

Warning

Calling this method between start/reset and run may result in a path to a database not yet created.
raises ValueError:
 if the system is in a state where path can’t be retrieved, for example before run is called
get_monitor_by_chip(description)[source]

Gets the monitor values for a specific item

Parameters:description (str) –
Returns:list of tuples x, y, value)
Return type:lits((int, int float))
get_provenance_for_router(x, y)[source]

Gets the provenance item(s) from the last run relating to a chip

Parameters:
  • x (int) – The X coordinate of the chip
  • y (int) – The Y coordinate of the chip
Returns:

A possibly multiline string with for each row which matches the like a line description_name: value

Return type:

str

get_router_by_chip(description)[source]

Gets the router values for a specific item

Parameters:description (str) –
Returns:list of tuples x, y, value)
Return type:lits((int, int float))
get_run_time_of_BufferExtractor()[source]

Gets the BufferExtractor provenance item(s) from the last run

Returns:A possibly multiline string with for each row which matches the like %BufferExtractor description_name: value
Return type:str
get_run_times()[source]

Gets the algorithm running times from the last run. If an algorithm is invoked multiple times in the run, its times are summed.

Returns:A possibly multiline string with for each row which matches the like a line description_name: time. The times are in seconds.
Return type:str
get_timer_provenance(algorithm)[source]

Gets the timer provenance item(s) from the last run

Parameters:algorithm (str) – The value to LIKE search for in the algorithm column. Can be the full name, or have % and _ wildcards.
Returns:A possibly multiline string with for each row which matches the like a line algorithm: value
Return type:str
get_timer_sum_by_algorithm(algorithm)[source]

Get the total runtime for one algorithm

Parameters:algorithm (str) –
Returns:total off all runtimes with this algorithm
Return type:int
get_timer_sum_by_category(category)[source]

Get the total runtime for one category of algorithms

Parameters:category (str) –
Returns:total off all runtimes with this category
Return type:int
messages()[source]

List all the provenance messages

Returns:all messages logged or not
Return type:list(str)
run_query(query, params=(), read_only=True, use_sqlite_rows=False)[source]

Opens a connection to the database, runs a query, extracts the results and closes the connection

The return type depends on the use_sqlite_rows param. By default this method returns tuples (lookup by index) but the advanced tuple type can be used instead, which supports lookup by name used in the query (use AS name in the query to set).

This method will not allow queries that change the database unless the read_only flag is set to False.

Note

This method is mainly provided as a support method for the later methods that return specific data. For new IntergationTests please add a specific method rather than call this directly.

Parameters:
Returns:

A list possibly empty of tuples/rows (one for each row in the database) where the number and type of the values corresponds to the where statement

Return type:

list(tuple or Row)

class spinn_front_end_common.interface.provenance.ProvenanceWriter(database_file=None, memory=False)[source]

Bases: spinn_front_end_common.utilities.sqlite_db.SQLiteDB

Specific implementation of the Database for SQLite 3.

Note

Not thread safe on the same database file. Threads can access different DBs just fine.

Note

This totally relies on the way SQLite’s type affinities function. You can’t port to a different database engine without a lot of work.

Parameters:
  • database_file (str or None) – The name of a file that contains (or will contain) an SQLite database holding the data. If omitted, either the default file path or an unshared in-memory database will be used (suitable only for testing).
  • memory (bool) – Flag to say unshared in-memory can be used. Otherwise a None file will mean the default should be used
add_core_name(x, y, p, core_name)[source]

Adds a vertex or similar name for the core to the core_mapping table

A second call to the same core is silently ignored even if the name if different.

Parameters:
  • x (int) – X coordinate of the chip
  • y (int) – Y coordinate of the chip
  • p (int) – id of the core
  • core_name (str) – Name to assign
insert_board_provenance(connections)[source]

Write the conection treived from spalloc job

Parameters:connections (dict((int, int): str) or None) – {(x, y): hostname, …} or None
insert_category_timing(category, timedelta, n_run, n_loop)[source]

Inserts algorithms run times into the timer_provenance table

Parameters:
  • category (str) – Category of the Algorithms run
  • timedelta (timedelta) – Time to be recorded
  • n_run (int) – The end user run number
  • n_loop (int or None) – The run loop within the ned user run
insert_connector(pre_population, post_population, the_type, description, the_value)[source]

Inserts edge data into the connector_provenance

Parameters:
  • pre_population (str) – Name of the pre population / vertex
  • post_population (str) – Name of the post population / vertex
  • the_type (str) – Class of the connector
  • description (str) – type of value
  • the_value (int) – data
insert_core(x, y, p, description, the_value)[source]

Inserts data for a specific core into the core_provenance table

Parameters:
  • x (int) – X coordinate of the chip
  • y (int) – Y coordinate of the chip
  • p (int) – id of the core
  • description (str) – type of value
  • the_value (int) – data
insert_gatherer(x, y, address, bytes, run, description, the_value)[source]

Records provenance into the gatherer_provenance

Parameters:
  • x (int) – X coordinate of the chip
  • y (int) – Y coordinate of the chip
  • address (int) – sdram address read from
  • bytes (int) – number of bytes read
  • run (int) – run number
  • description (str) – type of value
  • the_value (float) – data
insert_monitor(x, y, description, the_value)[source]

Inserts data into the monitor_provenance table

Parameters:
  • x (int) – X coordinate of the chip
  • y (int) – Y coordinate of the chip
  • description (str) – type of value
  • the_value (int) – data
insert_other(category, description, the_value)[source]

Insert unforeseen provenance into the other_provenace_table

This allows to add provenance that does not readily fit into any of the other categerogies

Parameters:
  • category (str) – grouping from this provenance
  • description (str) – Specific provenance being saved
  • the_value (ste) – Data
insert_power(description, the_value)[source]

Inserts a general power value into the power_provenane table

Parameters:
  • description (str) – Type of value
  • the_value (float) – data
insert_report(message)[source]

Save and if applicable logs a message to the report_table

Only logs the messages up to the cutoff set by cfg provenance_report_cutoff

Parameters:message (str) –
insert_router(x, y, description, the_value, expected=True)[source]

Inserts data into the router provenance table

Parameters:
  • x (int) – X coordinate of the chip
  • y (int) – Y coordinate of the chip
  • description (str) – type of value
  • the_value (float) – data
  • expected (bool) – Flag to say this data was expected
insert_timing(category, algorithm, the_value, n_run, n_loop, skip_reason)[source]

Inserts algorithms run times into the timer_provenance table

Parameters:
  • category (str) – Category of the Algorithm
  • algorithm (str) – Algorithm name
  • the_value (int) – Runtime
  • n_run (int) – The end user run number
  • n_loop (int or None) – The run loop within the ned user run
  • skip_reason – The reason the algorthm was skipped or None if it was not skipped
Tpye skip_reason:
 

str or None

insert_version(description, the_value)[source]

Inserts data into the version_provenance table

Parameters:
  • description (str) – The package for which the version applies
  • the_value (str) – The version to be recorded
class spinn_front_end_common.interface.provenance.ProvidesProvenanceDataFromMachineImpl[source]

Bases: spinn_front_end_common.interface.provenance.abstract_provides_provenance_data_from_machine.AbstractProvidesProvenanceDataFromMachine

An implementation that gets provenance data from a region of ints on the machine.

N_SYSTEM_PROVENANCE_WORDS = 6
class PROVENANCE_DATA_ENTRIES[source]

Bases: enum.Enum

Entries for the provenance data generated by models using provides provenance vertex.

CALLBACK_QUEUE_OVERLOADED = 1

The counter of the number of times the callback queue was overloaded

DMA_QUEUE_OVERLOADED = 2

The counter of the number of times the DMA queue was overloaded

MAX_NUMBER_OF_TIMER_TIC_OVERRUN = 5

The counter of the number of times the timer tick overran

TIMER_TIC_HAS_OVERRUN = 4

Whether the timer tick has overrun at all at any point

TRANSMISSION_EVENT_OVERFLOW = 0

The counter of transmission overflows

USER_QUEUE_OVERLOADED = 3

The counter of the number of times the user event queue overloaded

get_provenance_data_from_machine(transceiver, placement)[source]

Retrieve the provenance data.

Parameters:
  • transceiver (Transceiver) – How to talk to the machine
  • placement (Placement) – Which vertex are we retrieving from, and where was it
classmethod get_provenance_data_size(n_additional_data_items)[source]
Parameters:n_additional_data_items (int) –
Return type:int
parse_extra_provenance_items(label, x, y, p, provenance_data)[source]

Convert the remaining provenance words (those not in the standard set) into provenance items.

Called by get_provenance_data_from_machine()

Parameters:
  • label (str) – A descriptive label for the vertex (derived from label and placed position) to be used for provenance error reporting to the user.
  • x (int) – x coordinate of the chip where this core
  • y (int) – y coordinate of the core where this core
  • p (int) – virtual id of the core
  • provenance_data (list(int)) – The list of words of raw provenance data.
parse_system_provenance_items(label, x, y, p, provenance_data)[source]

Given some words of provenance data, convert the portion of them that describes the system provenance into proper provenance items.

Called by get_provenance_data_from_machine()

Parameters:
  • label (str) – A descriptive label for the vertex (derived from label and placed position) to be used for provenance error reporting to the user.
  • x (int) – x coordinate of the chip where this core
  • y (int) – y coordinate of the core where this core
  • p (int) – virtual id of the core
  • provenance_data (list(int)) –
reserve_provenance_data_region(spec)[source]
Parameters:spec (DataSpecificationGenerator) – The data specification being written.