subject

Abstraction layer around subject data storage files

Classes:

Subject(name, dir, file, structure[, data, ...])

Class for managing one subject's data and protocol.

class Subject(name: str | None = None, dir: ~pathlib.Path | None = None, file: ~pathlib.Path | None = None, structure: ~autopilot.data.models.subject.Subject_Structure = Subject_Structure(info=H5F_Group(path='/info', title='Subject Biographical Information', filters=None, attrs=None, children=None), data=H5F_Group(path='/data', title='', filters=Filters(complevel=6, complib='blosc:lz4', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None), attrs=None, children=None), protocol=H5F_Group(path='/protocol', title='Metadata for the currently assigned protocol', filters=None, attrs=None, children=None), history=H5F_Group(path='/history', title='', filters=None, attrs=None, children=[H5F_Group(path='/history/past_protocols', title='Past Protocol Files', filters=None, attrs=None, children=None), _Hash_Table(path='/history/hashes', title='Git commit hash history', filters=None, attrs=None, description=<class 'tables.description.Hashes'>, expectedrows=10000), _History_Table(path='/history/history', title='Change History', filters=None, attrs=None, description=<class 'tables.description.History'>, expectedrows=10000), _Weight_Table(path='/history/weights', title='Subject Weights', filters=None, attrs=None, description=<class 'tables.description.Weights'>, expectedrows=10000)])))[source]

Bases: object

Class for managing one subject’s data and protocol.

Creates a tables hdf5 file in prefs.get(‘DATADIR’) with the general structure:

/ root
|--- current (tables.filenode) storing the current task as serialized JSON
|--- data (group)
|    |--- task_name  (group)
|         |--- S##_step_name
|         |    |--- trial_data
|         |    |--- continuous_data
|         |--- ...
|--- history (group)
|    |--- hashes - history of git commit hashes
|    |--- history - history of changes: protocols assigned, params changed, etc.
|    |--- weights - history of pre and post-task weights
|    |--- past_protocols (group) - stash past protocol params on reassign
|         |--- date_protocol_name - tables.filenode of a previous protocol's params.
|         |--- ...
|--- info - group with biographical information as attributes
Variables:
  • name (str) – Subject ID

  • file (str) – Path to hdf5 file - usually {prefs.get(‘DATADIR’)}/{self.name}.h5

  • logger (logging.Logger) – from init_logger()

  • running (bool) – Flag that signals whether the subject is currently running a task or not.

  • data_queue (queue.Queue) – Queue to dump data while running task

  • did_graduate (threading.Event) – Event used to signal if the subject has graduated the current step

Parameters:
  • name (str) – subject ID

  • dir (str) – path where the .h5 file is located, if None, prefs.get(‘DATADIR’) is used

  • file (str) – load a subject from a filename. if None, ignored.

  • structure (Subject_Structure) – Structure to use with this subject.

Attributes:

info

Subject biographical information

bio

Subject biographical information (alias for info())

protocol

The status of the currently assigned protocol

protocol_name

Name of the currently assigned protocol

current_trial

Current number of trial for the assigned task

session

Current session of assigned protocol.

step

Current step of assigned protocol

task

Protocol dictionary for the current step

session_uuid

Automatically generated UUID given to each session, regardless of the session number.

history

The Subject's history of parameter and other changes.

hashes

History of version hashes and autopilot versions

weights

History of weights at the start and end of running a session.

Methods:

new(bio[, structure, data, attrs, children, ...])

Create a new subject file, make its structure, and populate its Biography .

update_history(type, name, value[, step])

Update the history table when changes are made to the subject's protocol.

assign_protocol(protocol[, step_n, pilot, ...])

Assign a protocol to the subject.

prepare_run()

Prepares the Subject object to receive data while running the task.

save_data(data)

Alternate and equivalent method of putting data in the queue as Subject.data_queue.put(data)

stop_run()

puts 'END' in the data_queue, which causes _data_thread() to end.

get_trial_data([step])

Get trial data from the current task.

get_weight([which, include_baseline])

Gets start and stop weights.

set_weight(date, col_name, new_value)

Updates an existing weight in the weight table.

update_weights([start, stop])

Store either a starting or stopping mass.

property info: Biography

Subject biographical information

property bio: Biography

Subject biographical information (alias for info())

property protocol: Protocol_Status | None

The status of the currently assigned protocol

See Protocol_Status

A property with an accompanying setter. When assigned to, stashes the details of the old protocol, and remakes the table structure to support the new task.

property protocol_name: str

Name of the currently assigned protocol

Convenience accessor for Subject.protocol.protocol_name

property current_trial: int

Current number of trial for the assigned task

Convenience accessor for .protocol.current_trial

Has Setter (can be assigned to)

property session: int

Current session of assigned protocol.

Convenience accessor for .protocol.session

Has setter (can be assigned to)

property step: int

Current step of assigned protocol

Convenience accessor for .protocol.step

Has setter (can be assigned to) to manually promote/demote subject to different steps of the protocol.

property task: dict

Protocol dictionary for the current step

property session_uuid: str

Automatically generated UUID given to each session, regardless of the session number.

Ensures each session is uniquely addressable in the case of ambiguous session numbers (eg. subject was manually promoted or demoted and session number was unable to be recovered, so there are multiple sessions with the same number)

property history: History

The Subject’s history of parameter and other changes.

See History

property hashes: Hashes

History of version hashes and autopilot versions

See Hashes

property weights: Weights

History of weights at the start and end of running a session.

See Weights

classmethod new(bio: ~autopilot.data.models.biography.Biography, structure: ~autopilot.data.models.subject.Subject_Structure | None = Subject_Structure(info=H5F_Group(path='/info', title='Subject Biographical Information', filters=None, attrs=None, children=None), data=H5F_Group(path='/data', title='', filters=Filters(complevel=6, complib='blosc:lz4', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None), attrs=None, children=None), protocol=H5F_Group(path='/protocol', title='Metadata for the currently assigned protocol', filters=None, attrs=None, children=None), history=H5F_Group(path='/history', title='', filters=None, attrs=None, children=[H5F_Group(path='/history/past_protocols', title='Past Protocol Files', filters=None, attrs=None, children=None), _Hash_Table(path='/history/hashes', title='Git commit hash history', filters=None, attrs=None, description=<class 'tables.description.Hashes'>, expectedrows=10000), _History_Table(path='/history/history', title='Change History', filters=None, attrs=None, description=<class 'tables.description.History'>, expectedrows=10000), _Weight_Table(path='/history/weights', title='Subject Weights', filters=None, attrs=None, description=<class 'tables.description.Weights'>, expectedrows=10000)])), path: ~pathlib.Path | None = None) Subject[source]

Create a new subject file, make its structure, and populate its Biography .

Parameters:
  • bio (Biography) – A collection of biographical information about the subject! Stored as attributes within /info

  • structure (Optional[Subject_Structure]) – The structure of tables and groups to use when creating this Subject. Note: This is not currently saved with the subject file, so if using a nonstandard structure, it needs to be passed every time on init. Sorry!

  • path (Optional[pathlib.Path]) – Path of created file. If None, make a file within the DATADIR within the user directory (typically ~/autopilot/data) using the subject ID as the filename. (eg. ~/autopilot/data/{id}.h5)

Returns:

Subject , Newly Created.

update_history(type, name: str, value: Any, step=None)[source]

Update the history table when changes are made to the subject’s protocol.

The current protocol is flushed to the past_protocols group and an updated filenode is created.

Note

This only updates the history table, and does not make the changes itself.

Parameters:
  • type (str) – What type of change is being made? Can be one of

    • ‘param’ - a parameter of one task stage

    • ‘step’ - the step of the current protocol

    • ‘protocol’ - the whole protocol is being updated.

  • name (str) – the name of either the parameter being changed or the new protocol

  • value (str) – the value that the parameter or step is being changed to, or the protocol dictionary flattened to a string.

  • step (int) – When type is ‘param’, changes the parameter at a particular step, otherwise the current step is used.

assign_protocol(protocol: Path | str | List[dict], step_n: int = 0, pilot: str | None = None, protocol_name: str | None = None)[source]

Assign a protocol to the subject.

If the subject has a currently assigned task, stashes it with stash_current()

Creates groups and tables according to the data descriptions in the task class being assigned. eg. as described in Task.TrialData.

Updates the history table.

Parameters:
  • protocol (Path, str, dict) – the protocol to be assigned. Can be one of

    • the name of the protocol (its filename minus .json) if it is in prefs.get(‘PROTOCOLDIR’)

    • filename of the protocol (its filename with .json) if it is in the prefs.get(‘PROTOCOLDIR’)

    • the full path and filename of the protocol.

    • The protocol dictionary serialized to a string

    • the protocol as a list of dictionaries

  • step_n (int) – Which step is being assigned?

  • protocol_name (str) – If passing protocol as a dict, have to give a name to the protocol

prepare_run() dict[source]

Prepares the Subject object to receive data while running the task.

Gets information about current task, trial number, spawns Graduation object, spawns data_queue and calls _data_thread().

Returns:

the parameters for the current step, with subject id, step number,

current trial, and session number included.

Return type:

Dict

save_data(data)[source]

Alternate and equivalent method of putting data in the queue as Subject.data_queue.put(data)

Parameters:

data (dict) – trial data. each should have a ‘trial_num’, and a dictionary with key ‘TRIAL_END’ should be passed at the end of each trial.

stop_run()[source]

puts ‘END’ in the data_queue, which causes _data_thread() to end.

get_trial_data(step: int | list | str | None = None) List[DataFrame] | DataFrame[source]

Get trial data from the current task.

Parameters:

step (int, list, str, None) – Step that should be returned, can be one of

  • None: All steps (default)

  • -1: the current step

  • int: a single step

  • list: of step numbers or step names (excluding S##_)

  • string: the name of a step (excluding S##_)

Returns:

DataFrame of requested steps’ trial data (or list of dataframes).

Return type:

pandas.DataFrame

get_weight(which='last', include_baseline=False)[source]

Gets start and stop weights.

Todo

add ability to get weights by session number, dates, and ranges.

Parameters:
  • which (str) – if ‘last’, gets most recent weights. Otherwise returns all weights.

  • include_baseline (bool) – if True, includes baseline and minimum mass.

Returns:

dict

set_weight(date, col_name, new_value)[source]

Updates an existing weight in the weight table.

Todo

Yes, i know this is bad. Merge with update_weights

Parameters:
  • date (str) – date in the ‘simple’ format, %y%m%d-%H%M%S

  • col_name (‘start’, ‘stop’) – are we updating a pre-task or post-task weight?

  • new_value (float) – New mass.

update_weights(start=None, stop=None)[source]

Store either a starting or stopping mass.

start and stop can be passed simultaneously, start can be given in one call and stop in a later call, but stop should not be given before start.

Parameters:
  • start (float) – Mass before running task in grams

  • stop (float) – Mass after running task in grams.