opihiexarata.opihi.database module

The database solution for storing and recording zero point data. We use a flat file database here with ordered directories to make it easily transversal by other tools and to have it be simple. A single instance of this class should monitor its own database.

class opihiexarata.opihi.database.OpihiZeroPointDatabaseSolution(database_directory: str)[source]

Bases: ExarataSolution

The flat file database solution which provides an API-like solution to interacting with a flat-file database with nested folders for all of the zero point entries.

database_directory

The path to the directory where the flat file database is.

Type:

string

__init__(database_directory: str) → None[source]

Creating the instance of the database for which we will use to manage the directory.

We check if the directory provided is a valid directory by checking for the existence of a file that indicates so. If the directory is blank, we assume a new database is to be created and so we add the file ourselves.

Parameters:

database_directory (string) – The path to the directory where the flat file database is. A database check file exists to ensure that it is a directory which follows the conventions of this class.

Return type:

None

_generate_text_record_filename(year: int, month: int, day: int) → str[source]

The text records which stores all of the information of the zero points are created in a specific way to allow for the user to manually navigate it with ease.

Parameters:

  • year (int) – The year of the data for this record.

  • month (int) – The month of the data for this record.

  • day (int) – The day of the data for this record.

Returns:

text_record_filename – The filename where the record should go in.

Return type:

string

_generate_zero_point_record_line(year: int, month: int, day: int, hour: int, minute: int, second: int, zero_point: float, zero_point_error: float, filter_name: str) → str[source]

This function creates the string representation of a zero point record.

Parameters:

  • year (int) – The year of the time of this zero point measurement.

  • month (int) – The month of the time of this zero point measurement.

  • day (int) – The day of the time of this zero point measurement.

  • hour (int) – The hour of the time of this zero point measurement.

  • minute (int) – The minute of the time of this zero point measurement.

  • second (int) – The second of the time of this zero point measurement.

  • zero_point (float) – The zero point measurement value

  • zero_point_error (float) – The error of the zero point measurement value.

  • filter_name (string) – The name of the filter that this zero point corresponds to.

Returns:

zero_point_record – The string representation of the zero point record.

Return type:

string

_parse_zero_point_record_line(record: str) → dict[source]

This function parses the zero point record line to return a dictionary which is much more helpful in working with the data.

Parameters:

record (string) – The record line which will be converted to a dictionary.

Returns:

record_dictionary – The record dictionary containing the information from the record line.

Return type:

dictionary

classmethod clean_database_text_files(database_directory: str, garbage_filename: str = None) → None[source]

This function cleans each and every file inside of the the database provided by the inputted database directory.

This is a class method because cleaning the database files does not rely on the database per-say or adding data to it. It could be performed by a third-party script without issue. It can also be dangerous to clean a database so we detach ourselves from the database we are managing just in case.

Parameters:

  • database_directory (str) – We clean all database files within the directory provided.

  • garbage_filename (str, default = None) – A path to a text file where all of the garbage lines detected by this cleaning routine should be saved to instead of being deleted as default.

Return type:

None

classmethod clean_record_text_file(filename: str, garbage_filename: str = None) → None[source]

This function takes a record file and cleans it up. Sorting it by time and deleting any lines which do not conform to the expected formatting along with any duplicate lines.

This is a class method because cleaning the database files does not rely on the database per-say or adding data to it. It could be performed by a third-party script without issue. It can also be dangerous to clean a database so we detach ourselves from the database we are managing just in case.

Parameters:

  • filename (str) – The text record filename which will have its entries cleaned up.

  • garbage_filename (str, default = None) – A path to a text file where all of the garbage lines detected by this cleaning routine should be saved to instead of being deleted as default.

Return type:

None

create_plotly_zero_point_html_plot(html_filename: str, plot_query_begin_jd: float, plot_query_end_jd: float, plot_lower_zero_point: float = None, plot_upper_zero_point: float = None, include_plotlyjs: str = True, using_timezone: str = None) → None[source]

This function creates the monitoring plot for the monitoring service webpage. It plots data from the zero point database depending on the range of times desires.

Parameters:

  • html_filename (string) – The filename where the html file will be saved to.

  • plot_query_begin_jd (float) – The starting time from which the database should be queried until for plotting. This is in Julian days as per convention.

  • plot_query_end_jd (float) – The starting time from which the database should be queried until for plotting. This is in Julian days as per convention.

  • plot_lower_zero_point (float, default = None) – This sets the lower limit of the zero point plot. If it and the upper limit is not set, we default to Plotly’s best judgement.

  • plot_upper_zero_point (float, default = None) – This sets the upper limit of the zero point plot. If it and the lower limit is not set, we default to Plotly’s best judgement.

  • include_plotlyjs (string, default = True) – The setting for how the plotly javascript file will be included. Consult the plotly documentation for available options.

  • using_timezone (string, default = None) – The timezone to use for the plot. By default, we use UTC, but, a different timezone can be provided to convert to.

Return type:

None

create_plotly_zero_point_html_plot_via_configuration() → None[source]

This is a wrapper function around create_plotly_zero_point_html_plot where the parameters of said function are supplied by the assumptions in the configuration file.

Namely, the duration that the database is queried is where the end query time is the current time (as of the function call) and the beginning time is some amount of hours ago. The resulting html plot file is saved to some location specified by the configuration file and the handling of the plotly javascript file is also detailed.

Parameters:

None –

Return type:

None

classmethod drop_database_text_files(database_directory: str) → None[source]

This function deletes all zero point record files within a given database. We use “drop” per database parlance.

This is a class method because deleting the database files does not rely on the database per-say or adding data to it. It could be performed by a third-party script without issue. It can also be dangerous to clean a database so we detach ourselves from the database we are managing just in case.

Parameters:

database_directory (str) – We delete all database files within the directory provided.

Return type:

None

query_database_all() → Table[source]

This queries the table for all of the data within it.

This is a wrapper function around the normal query tool for a range of dates which starts and ends way beyond the normal date ranges of Opihi, and so, will cover all of the data within the database.

Parameters:

None –

Returns:

query_record_table – A table containing the data as queried from the database across all reasonable times.

Return type:

Table

query_database_between_datetimes(begin_year: int, begin_month: int, begin_day: int, begin_hour: int, begin_minute: int, begin_second: int, end_year: int, end_month: int, end_day: int, end_hour: int, end_minute: int, end_second: int) → Table[source]

This queries the database and returns a table with all entries in the database in between two given dates and times.

Parameters:

  • begin_year (int) – Begin querying from this year of the date.

  • begin_month (int) – Begin querying from this month of the date.

  • begin_day (int) – Begin querying from this day of the date.

  • begin_hour (int) – Begin querying from this hour of the date.

  • begin_minute (int) – Begin querying from this minute of the date.

  • begin_second (int) – Begin querying from this second of the date.

  • end_year (int) – End querying from this year of the date.

  • end_month (int) – End querying from this month of the date.

  • end_day (int) – End querying from this day of the date.

  • end_hour (int) – End querying from this hour of the date.

  • end_minute (int) – End querying from this minute of the date.

  • end_second (int) – End querying from this second of the date.

Returns:

query_record_table – A table containing the data as queried from the database.

Return type:

Table

query_database_between_julian_days(begin_jd: float, end_jd: float) → Table[source]

This queries the database and returns a table with all entries in the database in between two given dates and times. The date and times are given in Julian days as per convention. This is a wrapper around the original implementation to account for the needed conversion.

Parameters:

  • begin_jd (float) – The Julian day after which, in time, records from the database should be returned.

  • end_jd (float) – The Julian day before which, in time, records from the database should be returned.

Returns:

query_record_table – A table containing the data as queried from the database.

Return type:

Table

read_zero_point_record_database() → Table[source]

This function reads all record files in the database that the instance is managing. Depending on the amount of data, this can be a little slow and take up a lot of memory, but it should be fine.

Parameters:

None –

Returns:

database_table – A table of all of the zero point data from files contained within the database.

Return type:

Table

read_zero_point_record_list(filename: str) → list[str][source]

This reads a zero point record file and converts it to a list for each row is a a row in the record file itself.

Parameters:

filename (string) – The zero point record filename to be read.

Returns:

record_list – The representation of all of the zero point data in a list.

Return type:

Table

read_zero_point_record_table(filename: str) → Table[source]

This reads a zero point record file and converts it into a nice table for easier reading and manipulation.

Parameters:

filename (string) – The zero point record filename to be read.

Returns:

record_table – The representation of all of the zero point data in a table.

Return type:

Table

write_zero_point_record(year: int, month: int, day: int, hour: int, minute: int, second: int, zero_point: float, zero_point_error: float, filter_name: str, clean_file: bool = False) → str[source]

This function writes a zero point measurement to the database.

Parameters:

  • year (int) – The year of the time of this zero point measurement.

  • month (int) – The month of the time of this zero point measurement.

  • day (int) – The day of the time of this zero point measurement.

  • hour (int) – The hour of the time of this zero point measurement.

  • minute (int) – The minute of the time of this zero point measurement.

  • second (int) – The second of the time of this zero point measurement.

  • zero_point (float) – The zero point measurement value

  • zero_point_error (float) – The error of the zero point measurement value.

  • filter_name (string) – The name of the filter that this zero point corresponds to.

  • clean_file (bool, default = False) – If True, along with appending the zero point record to the current file, we sort the file by time and clean up any bad entries.

Return type:

None

write_zero_point_record_julian_day(jd: float, zero_point: float, zero_point_error: float, filter_name: str, clean_file: bool = False) → None[source]

This function writes a zero point measurement to the database. However, it is also a wrapper around the standard function to allow for inputting times as Julian days, the convention for the OpihiExarata software.

Parameters:

  • jd (float) – The time of the zero point measurement, in Julian days.

  • zero_point (float) – The zero point of the measurement.

  • zero_point_error (float) – The error of the zero point measurement.

  • filter_name (float) – The name of the filter that the zero point measurement corresponds to.

  • clean_file (bool, default = False) – If True, along with appending the zero point record to the current file, we sort the file by time and clean up any bad entries.

Return type:

None