subject
Abstraction layer around subject data storage files
Classes:
|
Class for managing one subject's data and protocol. |
- class Subject(name: str | None = None, dir: ~pathlib.Path | None = None, file: ~pathlib.Path | None = None, structure: ~autopilot.data.models.subject.Subject_Structure = Subject_Structure(info=H5F_Group(path='/info', title='Subject Biographical Information', filters=None, attrs=None, children=None), data=H5F_Group(path='/data', title='', filters=Filters(complevel=6, complib='blosc:lz4', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None), attrs=None, children=None), protocol=H5F_Group(path='/protocol', title='Metadata for the currently assigned protocol', filters=None, attrs=None, children=None), history=H5F_Group(path='/history', title='', filters=None, attrs=None, children=[H5F_Group(path='/history/past_protocols', title='Past Protocol Files', filters=None, attrs=None, children=None), _Hash_Table(path='/history/hashes', title='Git commit hash history', filters=None, attrs=None, description=<class 'tables.description.Hashes'>, expectedrows=10000), _History_Table(path='/history/history', title='Change History', filters=None, attrs=None, description=<class 'tables.description.History'>, expectedrows=10000), _Weight_Table(path='/history/weights', title='Subject Weights', filters=None, attrs=None, description=<class 'tables.description.Weights'>, expectedrows=10000)])))[source]
Bases:
object
Class for managing one subject’s data and protocol.
Creates a
tables
hdf5 file in prefs.get(‘DATADIR’) with the general structure:/ root |--- current (tables.filenode) storing the current task as serialized JSON |--- data (group) | |--- task_name (group) | |--- S##_step_name | | |--- trial_data | | |--- continuous_data | |--- ... |--- history (group) | |--- hashes - history of git commit hashes | |--- history - history of changes: protocols assigned, params changed, etc. | |--- weights - history of pre and post-task weights | |--- past_protocols (group) - stash past protocol params on reassign | |--- date_protocol_name - tables.filenode of a previous protocol's params. | |--- ... |--- info - group with biographical information as attributes
- Variables:
name (str) – Subject ID
file (str) – Path to hdf5 file - usually {prefs.get(‘DATADIR’)}/{self.name}.h5
logger (
logging.Logger
) – frominit_logger()
running (bool) – Flag that signals whether the subject is currently running a task or not.
data_queue (
queue.Queue
) – Queue to dump data while running taskdid_graduate (
threading.Event
) – Event used to signal if the subject has graduated the current step
- Parameters:
name (str) – subject ID
dir (str) – path where the .h5 file is located, if None, prefs.get(‘DATADIR’) is used
file (str) – load a subject from a filename. if None, ignored.
structure (
Subject_Structure
) – Structure to use with this subject.
Attributes:
Subject biographical information
Subject biographical information (alias for
info()
)The status of the currently assigned protocol
Name of the currently assigned protocol
Current number of trial for the assigned task
Current session of assigned protocol.
Current step of assigned protocol
Protocol dictionary for the current step
Automatically generated UUID given to each session, regardless of the session number.
The Subject's history of parameter and other changes.
History of version hashes and autopilot versions
History of weights at the start and end of running a session.
Methods:
new
(bio[, structure, data, attrs, children, ...])Create a new subject file, make its structure, and populate its
Biography
.update_history
(type, name, value[, step])Update the history table when changes are made to the subject's protocol.
assign_protocol
(protocol[, step_n, pilot, ...])Assign a protocol to the subject.
Prepares the Subject object to receive data while running the task.
save_data
(data)Alternate and equivalent method of putting data in the queue as Subject.data_queue.put(data)
stop_run
()puts 'END' in the data_queue, which causes
_data_thread()
to end.get_trial_data
([step])Get trial data from the current task.
get_weight
([which, include_baseline])Gets start and stop weights.
set_weight
(date, col_name, new_value)Updates an existing weight in the weight table.
update_weights
([start, stop])Store either a starting or stopping mass.
- property protocol: Protocol_Status | None
The status of the currently assigned protocol
See
Protocol_Status
A property with an accompanying setter. When assigned to, stashes the details of the old protocol, and remakes the table structure to support the new task.
- property protocol_name: str
Name of the currently assigned protocol
Convenience accessor for
Subject.protocol.protocol_name
- property current_trial: int
Current number of trial for the assigned task
Convenience accessor for
.protocol.current_trial
Has Setter (can be assigned to)
- property session: int
Current session of assigned protocol.
Convenience accessor for
.protocol.session
Has setter (can be assigned to)
- property step: int
Current step of assigned protocol
Convenience accessor for
.protocol.step
Has setter (can be assigned to) to manually promote/demote subject to different steps of the protocol.
- property session_uuid: str
Automatically generated UUID given to each session, regardless of the session number.
Ensures each session is uniquely addressable in the case of ambiguous session numbers (eg. subject was manually promoted or demoted and session number was unable to be recovered, so there are multiple sessions with the same number)
- property weights: Weights
History of weights at the start and end of running a session.
See
Weights
- classmethod new(bio: ~autopilot.data.models.biography.Biography, structure: ~autopilot.data.models.subject.Subject_Structure | None = Subject_Structure(info=H5F_Group(path='/info', title='Subject Biographical Information', filters=None, attrs=None, children=None), data=H5F_Group(path='/data', title='', filters=Filters(complevel=6, complib='blosc:lz4', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None), attrs=None, children=None), protocol=H5F_Group(path='/protocol', title='Metadata for the currently assigned protocol', filters=None, attrs=None, children=None), history=H5F_Group(path='/history', title='', filters=None, attrs=None, children=[H5F_Group(path='/history/past_protocols', title='Past Protocol Files', filters=None, attrs=None, children=None), _Hash_Table(path='/history/hashes', title='Git commit hash history', filters=None, attrs=None, description=<class 'tables.description.Hashes'>, expectedrows=10000), _History_Table(path='/history/history', title='Change History', filters=None, attrs=None, description=<class 'tables.description.History'>, expectedrows=10000), _Weight_Table(path='/history/weights', title='Subject Weights', filters=None, attrs=None, description=<class 'tables.description.Weights'>, expectedrows=10000)])), path: ~pathlib.Path | None = None) Subject [source]
Create a new subject file, make its structure, and populate its
Biography
.- Parameters:
bio (
Biography
) – A collection of biographical information about the subject! Stored as attributes within /infostructure (Optional[
Subject_Structure
]) – The structure of tables and groups to use when creating this Subject. Note: This is not currently saved with the subject file, so if using a nonstandard structure, it needs to be passed every time on init. Sorry!path (Optional[
pathlib.Path
]) – Path of created file. IfNone
, make a file within theDATADIR
within the user directory (typically~/autopilot/data
) using the subject ID as the filename. (eg.~/autopilot/data/{id}.h5
)
- Returns:
Subject
, Newly Created.
- update_history(type, name: str, value: Any, step=None)[source]
Update the history table when changes are made to the subject’s protocol.
The current protocol is flushed to the past_protocols group and an updated filenode is created.
Note
This only updates the history table, and does not make the changes itself.
- Parameters:
type (str) – What type of change is being made? Can be one of
‘param’ - a parameter of one task stage
‘step’ - the step of the current protocol
‘protocol’ - the whole protocol is being updated.
name (str) – the name of either the parameter being changed or the new protocol
value (str) – the value that the parameter or step is being changed to, or the protocol dictionary flattened to a string.
step (int) – When type is ‘param’, changes the parameter at a particular step, otherwise the current step is used.
- assign_protocol(protocol: Path | str | List[dict], step_n: int = 0, pilot: str | None = None, protocol_name: str | None = None)[source]
Assign a protocol to the subject.
If the subject has a currently assigned task, stashes it with
stash_current()
Creates groups and tables according to the data descriptions in the task class being assigned. eg. as described in
Task.TrialData
.Updates the history table.
- Parameters:
protocol (Path, str, dict) – the protocol to be assigned. Can be one of
the name of the protocol (its filename minus .json) if it is in prefs.get(‘PROTOCOLDIR’)
filename of the protocol (its filename with .json) if it is in the prefs.get(‘PROTOCOLDIR’)
the full path and filename of the protocol.
The protocol dictionary serialized to a string
the protocol as a list of dictionaries
step_n (int) – Which step is being assigned?
protocol_name (str) – If passing
protocol
as a dict, have to give a name to the protocol
- prepare_run() dict [source]
Prepares the Subject object to receive data while running the task.
Gets information about current task, trial number, spawns
Graduation
object, spawnsdata_queue
and calls_data_thread()
.- Returns:
- the parameters for the current step, with subject id, step number,
current trial, and session number included.
- Return type:
Dict
- save_data(data)[source]
Alternate and equivalent method of putting data in the queue as Subject.data_queue.put(data)
- Parameters:
data (dict) – trial data. each should have a ‘trial_num’, and a dictionary with key ‘TRIAL_END’ should be passed at the end of each trial.
- get_trial_data(step: int | list | str | None = None) List[DataFrame] | DataFrame [source]
Get trial data from the current task.
- Parameters:
step (int, list, str, None) – Step that should be returned, can be one of
None
: All steps (default)-1: the current step
int: a single step
list: of step numbers or step names (excluding S##_)
string: the name of a step (excluding S##_)
- Returns:
DataFrame of requested steps’ trial data (or list of dataframes).
- Return type:
- get_weight(which='last', include_baseline=False)[source]
Gets start and stop weights.
Todo
add ability to get weights by session number, dates, and ranges.
- Parameters:
which (str) – if ‘last’, gets most recent weights. Otherwise returns all weights.
include_baseline (bool) – if True, includes baseline and minimum mass.
- Returns:
dict
- set_weight(date, col_name, new_value)[source]
Updates an existing weight in the weight table.
Todo
Yes, i know this is bad. Merge with update_weights
- Parameters:
date (str) – date in the ‘simple’ format, %y%m%d-%H%M%S
col_name (‘start’, ‘stop’) – are we updating a pre-task or post-task weight?
new_value (float) – New mass.
- update_weights(start=None, stop=None)[source]
Store either a starting or stopping mass.
start and stop can be passed simultaneously, start can be given in one call and stop in a later call, but stop should not be given before start.
- Parameters:
start (float) – Mass before running task in grams
stop (float) – Mass after running task in grams.