Base classes for storage engines
panko.storage.base.
Connection
(conf)¶Bases: object
Base class for event storage system connections.
CAPABILITIES
= {'events': {'query': {'simple': False}}}¶STORAGE_CAPABILITIES
= {'storage': {'production_ready': False}}¶clear
()¶Clear database.
clear_expired_data
(ttl, max_count=None)¶Clear expired data from the backend storage system.
Clearing occurs according to the time-to-live. :param ttl: Number of seconds to keep records for. :param max_count: Number of records to delete.
get_capabilities
()¶Return an dictionary with the capabilities of each driver.
get_event_types
()¶Return all event types as an iterable of strings.
get_events
(event_filter, pagination=None)¶Return an iterable of model.Event objects.
get_storage_capabilities
()¶Return a dictionary representing the performance capabilities.
This is needed to evaluate the performance of each driver.
get_trait_types
(event_type)¶Return a dictionary containing the name and data type of the trait.
Only trait types for the provided event_type are returned. :param event_type: the type of the Event
get_traits
(event_type, trait_type=None)¶Return all trait instances associated with an event_type.
If trait_type is specified, only return instances of that trait type. :param event_type: the type of the Event to filter by :param trait_type: the name of the Trait to filter by
record_events
(events)¶Write the events to the backend storage system.
events – a list of model.Event objects.
upgrade
()¶Migrate the database to version or the most recent version.
panko.storage.impl_elasticsearch.
Connection
(url, conf)¶Bases: panko.storage.base.Connection
Put the event data into an ElasticSearch db.
Events in ElasticSearch are indexed by day and stored by event_type. An example document:
{"_index":"events_2014-10-21",
"_type":"event_type0",
"_id":"dc90e464-65ab-4a5d-bf66-ecb956b5d779",
"_score":1.0,
"_source":{"timestamp": "2014-10-21T20:02:09.274797"
"traits": {"id4_0": "2014-10-21T20:02:09.274797",
"id3_0": 0.7510790937279408,
"id2_0": 5,
"id1_0": "18c97ba1-3b74-441a-b948-a702a30cbce2"}
}
}
CAPABILITIES
= {'events': {'query': {'simple': True}}}¶STORAGE_CAPABILITIES
= {'storage': {'production_ready': True}}¶get_event_types
()¶Return all event types as an iterable of strings.
get_events
(event_filter, pagination=None)¶Return an iterable of model.Event objects.
get_trait_types
(event_type)¶Return a dictionary containing the name and data type of the trait.
Only trait types for the provided event_type are returned. :param event_type: the type of the Event
get_traits
(event_type, trait_type=None)¶Return all trait instances associated with an event_type.
If trait_type is specified, only return instances of that trait type. :param event_type: the type of the Event to filter by :param trait_type: the name of the Trait to filter by
record_events
(events)¶Write the events to the backend storage system.
events – a list of model.Event objects.
upgrade
()¶Migrate the database to version or the most recent version.
panko.storage.impl_log.
Connection
(conf)¶Bases: panko.storage.base.Connection
Log event data.
clear_expired_data
(ttl, max_count)¶Clear expired data from the backend storage system.
Clearing occurs according to the time-to-live.
ttl – Number of seconds to keep records for.
max_count – Number of records to delete.
MongoDB storage backend
panko.storage.impl_mongodb.
Connection
(url, conf)¶Bases: panko.storage.pymongo_base.Connection
Put the event data into a MongoDB database.
CONNECTION_POOL
= <panko.storage.mongo.utils.ConnectionPool object>¶clear
()¶Clear database.
clear_expired_data
(ttl, max_count=None)¶Clear expired data from the backend storage system.
Clearing occurs according to the time-to-live.
ttl – Number of seconds to keep records for.
max_count – Number of records to delete (not used for MongoDB).
update_ttl
(ttl, ttl_index_name, index_field, coll)¶Update or create time_to_live indexes.
ttl – time to live in seconds.
ttl_index_name – name of the index we want to update or create.
index_field – field with the index that we need to update.
coll – collection which indexes need to be updated.
upgrade
()¶Migrate the database to version or the most recent version.
SQLAlchemy storage backend.
panko.storage.impl_sqlalchemy.
Connection
(url, conf)¶Bases: panko.storage.base.Connection
Put the event data into a SQLAlchemy database.
Tables:
- EventType
- event definition
- { id: event type id
desc: description of event
}
- Event
- event data
- { id: event id
message_id: message id
generated = timestamp of event
event_type_id = event type -> eventtype.id
}
- TraitInt
- int trait value
- { event_id: event -> event.id
key: trait name
value: integer value
}
- TraitDatetime
- datetime trait value
- { event_id: event -> event.id
key: trait name
value: datetime value
}
- TraitText
- text trait value
- { event_id: event -> event.id
key: trait name
value: text value
}
- TraitFloat
- float trait value
- { event_id: event -> event.id
key: trait name
value: float value
}
CAPABILITIES
= {'events': {'query': {'simple': True}}}¶STORAGE_CAPABILITIES
= {'storage': {'production_ready': True}}¶clear
()¶Clear database.
clear_expired_data
(ttl, max_count)¶Clear expired data from the backend storage system.
Clearing occurs according to the time-to-live.
ttl – Number of seconds to keep records for.
max_count – Number of records to delete.
dress_url
(url)¶get_event_types
()¶Return all event types as an iterable of strings.
get_events
(event_filter, pagination=None)¶Return an iterable of model.Event objects.
event_filter – EventFilter instance
pagination – Pagination parameters.
get_trait_types
(event_type)¶Return a dictionary containing the name and data type of the trait.
Only trait types for the provided event_type are returned. :param event_type: the type of the Event
get_traits
(event_type, trait_type=None)¶Return all trait instances associated with an event_type.
If trait_type is specified, only return instances of that trait type. :param event_type: the type of the Event to filter by :param trait_type: the name of the Trait to filter by
record_events
(event_models)¶Write the events to SQL database via sqlalchemy.
event_models – a list of model.Event objects.
upgrade
()¶Migrate the database to version or the most recent version.
Model classes for use in the events storage API.
panko.storage.models.
Event
(message_id, event_type, generated, traits, raw)¶Bases: panko.storage.base.Model
A raw event from the source system. Events have Traits.
Metrics will be derived from one or more Events.
DEFAULT_DIR
= 'asc'¶DEFAULT_SORT
= [('generated', 'asc'), ('message_id', 'asc')]¶DUPLICATE
= 1¶INCOMPATIBLE_TRAIT
= 3¶PRIMARY_KEY
= 'message_id'¶SUPPORT_DIRS
= ('asc', 'desc')¶SUPPORT_SORT_KEYS
= ('message_id', 'generated')¶UNKNOWN_PROBLEM
= 2¶append_trait
(trait_model)¶serialize
()¶panko.storage.models.
Trait
(name, dtype, value)¶Bases: panko.storage.base.Model
A Trait is a key/value pair of data on an Event.
The value is variant record of basic data types (int, date, float, etc).
DATETIME_TYPE
= 4¶FLOAT_TYPE
= 3¶INT_TYPE
= 2¶NONE_TYPE
= 0¶TEXT_TYPE
= 1¶convert_value
(trait_type, value)¶get_name_by_type
(type_id)¶get_type_by_name
(type_name)¶get_type_name
()¶get_type_names
()¶serialize
()¶type_names
= {0: 'none', 1: 'string', 2: 'integer', 3: 'float', 4: 'datetime'}¶panko.storage.models.
serialize_dt
(value)¶Serializes parameter if it is datetime.
Common functions for MongoDB backend
panko.storage.pymongo_base.
Connection
(conf)¶Bases: panko.storage.base.Connection
Base event Connection class for MongoDB driver.
CAPABILITIES
= {'events': {'query': {'simple': True}}}¶STORAGE_CAPABILITIES
= {'storage': {'production_ready': True}}¶get_event_types
()¶Return all event types as an iter of strings.
get_events
(event_filter, pagination=None)¶Return an iter of models.Event objects.
event_filter – storage.EventFilter object, consists of filters for events that are stored in database.
pagination – Pagination parameters.
get_trait_types
(event_type)¶Return a dictionary containing the name and data type of the trait.
Only trait types for the provided event_type are returned.
event_type – the type of the Event.
get_traits
(event_type, trait_name=None)¶Return all trait instances associated with an event_type.
If trait_type is specified, only return instances of that trait type.
event_type – the type of the Event to filter by
trait_name – the name of the Trait to filter by
record_events
(event_models)¶Write the events to database.
event_models – a list of models.Event objects.
Storage backend management
panko.storage.
EventFilter
(start_timestamp=None, end_timestamp=None, event_type=None, message_id=None, traits_filter=None, admin_proj=None)¶Bases: object
Properties for building an Event query.
start_timestamp – UTC start datetime (mandatory)
end_timestamp – UTC end datetime (mandatory)
event_type – the name of the event. None for all.
message_id – the message_id of the event. None for all.
admin_proj – the project_id of admin role. None if non-admin user.
traits_filter – the trait filter dicts, all of which are optional. This parameter is a list of dictionaries that specify trait values:
{'key': <key>,
'string': <value>,
'integer': <value>,
'datetime': <value>,
'float': <value>,
'op': <eq, lt, le, ne, gt or ge> }
panko.storage.
InvalidMarker
¶Bases: Exception
Invalid pagination marker parameters
panko.storage.
StorageBadAggregate
¶Bases: Exception
Error raised when an aggregate is unacceptable to storage backend.
code
= 400¶panko.storage.
StorageBadVersion
¶Bases: Exception
Error raised when the storage backend version is not good enough.
panko.storage.
StorageUnknownWriteError
¶Bases: Exception
Error raised when an unknown error occurs while recording.
panko.storage.
get_connection
(url, conf)¶Return an open connection to the database.
panko.storage.
get_connection_from_config
(conf)¶Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.