hypertunity.reports

Summary

Default

Reporter	Abstract class Reporter for result visualisation.
Table	A Reporter subclass to print and store a formatted table of the results.

Optional

tensorboard.Tensorboard

A Reporter subclass to visualise the results in Tensorboard.

API documentation

class Reporter(domain, metrics, primary_metric='', database_path=None)

Abstract class Reporter for result visualisation.

__init__(domain, metrics, primary_metric='', database_path=None)

Initialise the base reporter with domain and metrics.

Parameters:

• domain – A Domain from which all evaluated samples are drawn.
• metrics – List[str] with names of the metrics used during evaluation.
• primary_metric – (optional) str primary metric from metrics. This is used to determine the best sample. Defaults to the first one.
• database_path – (optional) str path to the database for storing experiment history on disk. Defaults to in-memory storage.

property database

Return the logging database.

property default_database_table

Return the default database table name.

from_database(database, table=None)

Load history from a database supplied as a path to a file or a tinydb.TinyDB object.

Parameters:

• database – str or tinydb.TinyDB. The database to load.
• table – (optional) str. The table to load from the database. This argument is not required if the database has only one table.

Raises:

ValueError – if the database contains more than one table and table is not given.

from_history(history)

Load the reporter with data from an entry of evaluations.

Parameters:history – List[HistoryPoint] or Tuple. The sequence of evaluations comprised of samples and metrics.

get_best(criterion='max')

Return the entry from the database which corresponds to the best scoring experiment.

Parameters:criterion – str or Callable. The function used to determine whether the highest or lowest score is requested. If several evaluation metrics are present, then a custom criterion must be supplied. Returns:JSON object or None if the database is empty. The content of the database for the best experiment.

log(entry, **kwargs)

Create an entry for an optimisation history point in the Reporter.

Parameters:

• entry – HistoryPoint or Tuple[Sample, Dict]. The history point to log. If given as a tuple of Sample instance and a mapping from metric names to results, the variance of the evaluation noise can be supplied by adding an entry in the dict with the metric name and the suffix ‘_var’.
• **kwargs – (optional) Any. Additional arguments for the logging implementation in a subclass.

Keyword Arguments:  

meta – (optional) additional information to be logged in the database for this entry.

to_history(table=None)

Export the reporter logged history from a database table to an optimiser-friendly history.

Parameters:table – (optional) str. The name of the table to export. Defaults to the one created during reporter initialisation. Returns:A list of HistoryPoint objects which can be loaded into an Optimiser instance.

class Table(domain, metrics, primary_metric='', database_path=None)

A Reporter subclass to print and store a formatted table of the results.

__init__(domain, metrics, primary_metric='', database_path=None)

Initialise the table reporter with domain and metrics.

Parameters:

• domain – A Domain from which all evaluated samples are drawn.
• metrics – List[str] with names of the metrics used during evaluation.
• primary_metric – (optional) str primary metric from metrics. This is used to determine the best sample. Defaults to the first one.
• database_path – (optional) str path to the database for storing experiment history on disk. Defaults to in-memory storage.

__str__()

Return the string representation of the table.

property data

Return the table as a numpy array.

format(order='none', emphasise=False)

Format the table and return it as a string.

Supported formatting is sorting and emphasising of the best result.

Parameters:

• order – (optional) str. The order of sorting by the primary metric. Can be “none”, “ascending” or “descending”. Defaults to “none”.
• emphasise – (optional) bool. Whether to emphasise the best experiment by marking it in yellow and blinking if supported. Defaults to False.

Returns:

str of the formatted table.

from_database(database, table=None)

Load history from a database supplied as a path to a file or a tinydb.TinyDB object.

Parameters:

• database – str or tinydb.TinyDB. The database to load.
• table – (optional) str. The table to load from the database. This argument is not required if the database has only one table.

Raises:

• ValueError – if the database contains more than one table
• and table is not given. –

class Tensorboard(domain, metrics, logdir, primary_metric='', database_path=None)

A Reporter subclass to visualise the results in Tensorboard.

It utilises Tensorboard’s HParams plugin as a dashboard for the summary of the optimisation. This class prepares and creates entries with the scalar data of the experiment trials, containing the domain sample and the corresponding metrics.

Notes

The user is responsible for launching TensorBoard in the browser.

__init__(domain, metrics, logdir, primary_metric='', database_path=None)

Initialise the TensorBoard reporter.

Parameters:

• domain – Domain. The domain to which all evaluated samples belong.
• metrics – List[str]. The names of the metrics.
• logdir – str. Path to a folder for storing the Tensorboard events.
• primary_metric – (optional) str. Primary metric from metrics. This is used by the format() method to determine the sorting column and the best value. Default is the first one.
• database_path – (optional) str. The path to the database for storing experiment history on disk. Default is in-memory storage.

from_database(database, table=None)

Load history from a database supplied as a path to a file or a tinydb.TinyDB object.

Parameters:

• database – str or tinydb.TinyDB. The database to load.
• table – (optional) str. The table to load from the database. This argument is not required if the database has only one table.

Raises:

• ValueError – if the database contains more than one table
• and table is not given. –