v0.3.5 (February 22, 2021)

Bugfixes

  • Very minor one, fixes to the way Terminal accesses the pilot_db.json file to use Terminal.pilots property that makes a new pilot_db.json file if one doesn’t exist, but otherwise loads the one that is found in prefs.get('PILOT_DB')

  • Reorganized Terminal source to group properties together & minor additions of type hinting

  • Fixed some bad fallback behavior looking for files in old hardcoded default directories, eg. in the ye olde utils.get_pilotdb()

v0.3.4 (December 13, 2020)

Improvements

Bugfixes

Docs

Logging

v0.3.3 (October 25, 2020)

Bugfixes

  • Fix layout in batch reassign gui widget from python 3 float division

  • Cleaner close by catching KeyboardInterrupt in networking modules

  • Fixing audioserver boot options – if ‘AUDIOSERVER’ is set even if ‘AUDIO’ isn’t set in prefs, should still start server. Not full fixed, need to make single plugin handler, single point of enabling/disabling optional services like audio server

  • Fix conflict between polarity and pull in initializing pulls in pilot

  • Catch tables.HDF5ExtError if local .h5 file corrupt in pilot

  • For some reason ‘fs’ wasn’t being replaced in the jackd string, reinstated.

  • Fix comparison in LED_RGB that caused ‘0’ to turn on full becuse ‘value’ was being checked for its truth value (0 is false) rather than checking if value is None.

  • obj.next() to next(obj)` in jackdserver

Improvements

  • Better internal handling of pigpiod – you’re now able to import and use hardware modules without needing to explicitly start pigpiod!!

  • Hopefully better killing of processes on exit, though still should work into unified process manager so don’t need to reimplement everything (eg. as is done with launching pigpiod and jackd)

  • Environment scripts have been split out into setup/scripts.py and you can now run them with python -m autopilot.setup.run_script (use --help to see how!)

  • Informative error when setup is run with too narrow terminal: https://github.com/wehr-lab/autopilot/issues/23

  • More loggers, but increased need to unify logger creation!!!

Cleanup

  • remove unused imports in main __init__.py that made cyclical imports happen more frequently than necessary

  • single-sourcing version number from __init__.py

  • more cleanup of unnecessary meta and header stuff left from early days

  • more debugging flags

  • filter NaturalNameWarning from pytables

  • quieter cleanups for hardware objects

v0.3.2 (September 28, 2020)

Bugfixes

  • https://github.com/wehr-lab/autopilot/issues/19 - previously, I attempted to package binaries for the lightly modified pigpio and for jackd (the apt binary used to not work), but after realizing that was the worst possible way of going about it I changed install strategies, but didn’t entirely remove the vestiges of the prior attempt. The installation expected certain directories to exist (in autopilot/external) that didn’t, which crashed and choked install. Still need to formalize a configuration and plugin system, but getting there.

  • https://github.com/wehr-lab/autopilot/issues/20 - the jackd binary in the apt repos for the raspi used to not work, so i was in the habit of compiling jackd audio from source. I had build that into the install routine, but something about that now causes the JACK-Client python interface to throw segfaults. Somewhere along the line someone fixed the apt repo version of jackd so we use that now.

  • previously I had only tested in a virtual environment, but now the installation routine properly handles not being in a venv.

Cleanup

  • remove bulky static files like fonts and css from /docs/ where they were never needed and god knows how they got there

  • use a forked sphinx-sass when building docs that doesn’t specify a required sphinx version (which breaks sphinx)

  • removed skbuild requirements from install

  • fixed pigpio install requirement in requirements_pilot.txt

  • included various previously missed files in MANIFEST.in

  • added installation of system libraries to the pilot configuration menu

v0.3.1 (August 4, 2020)

Practice version!!! still figuring out pypi

v0.3.0 (August 4, 2020)

Major Updates

  • Python 3 - We’ve finally made it to Python 3! Specifically we have brought Autopilot up to compatibility with Python 3.8 – though the Spinnaker SDK is currently only available through Python 3.7, so we have formally required 3.7 for now while we work on moving acquisition to Aravis. I will not attempt to keep Autopilot compatible with Python 2, but no decision has been made about compatibility with other versions of Python 3. Until then, expect that Autopilot will attempt to keep up with major version changes. The switch also let up update PySide (Qt library used for the GUI) to PySide2, which uses Qt5 and has a whole raft of other improvements.

  • Continuous Data Handling - The Subject class and networking modules have been improved to handle continuous data (eg. streaming data, generally non-trialwise or non-event-sampled data). Continuous data can be set in a Task description either with a tables column descriptor as trial data is, but also can be set as 'infer', for which the Subject class will wait until it receives the first data and automatically create a tables column depending on its type and shape. While previously we intended to nudge users to be explicit about declaring their data, this was necessary to allow for data that might be variable in type and shape to be included in a Task – eg. it should be possible to record video data without needing to specify the resolution or bit depth as a hardcoded parameter in a task class. I have come to like type inference, and may make it a general practice for all types of data. That would potentially allow tasks to be written without explicitly declaring the data that they produce at all, but I haven’t decided if that’s a good thing or not yet.

  • The GPIO engine has been rebuilt, relying more on pigpio’s function interface. This means that GPIO timing is now ~microsecond precise, important for reward delivery, LED flashing, and a number of other basic infrastructural needs. The reorganization of hardware modules resulted in general GPIO, Digital_In and Digital_Out metaclasses, making common operations like setting polarity, triggers, and pullup/down resistors much easier.

  • Setup has been greatly improved. This includes proper packaging and installation with setuptools & sk-build, allowing us to finally join PyPI :) https://pypi.org/project/auto-pi-lot/ . Setup has been unified into a single npyscreen-based set of prompts that allow the user to run scripts to install libraries or configure their environment (also see run_script() and list_scripts()), set prefs, configure hardware objects (based on some very fun signature introspection), setup autopilot as a systemd service, etc. Getting started with Autopilot is now three commands!:

    pip install auto-pi-lot
autopilot.setup.setup_autopilot
~/autopilot/launch_autopilot.sh

Minor Updates

  • Logging level is now set from prefs, so where before, eg. every message through the networking modules would be logged to stdout, now only warnings and exceptions are. This gives a surprisingly large performance boost.

    • Logging has also been much improved in networking modules, where rather than an awkward do_logging flag that was used to avoid logging performance-critical events like streaming data, logging is controlled by log level throughout the system. By default, logging of most messages is set at debug level so they don’t drown out important messages in the logs as they used to.

  • Networking modules now only deserialize messages if they are the final recipient, saving lots of processing time – particularly with streamed arrays. Message objects also only re-serialize messages if they have been changed. Message structure has been changed such that serialized messages are now of the general format:

    [sender,
 (optional) intermediate_node_1, intermediate_node_2, ...
 final_recipient,
 message_contents]

  • Configuration will continue to be a point of improvement, but a few minor updates were made:

    • prefs.CONFIG will be used to signal multiple, potentially overlapping agent configurations, each of which may have their own system dependencies, external daemons, etc. Eg. a Pilot could be configured to play audio (which requires a jackd daemon to be started before Autopilot) and video (which requires Autopilot to be started in a X session). Checks of prefs.CONFIG are now in rather than == to reflect that.

    • prefs.PINS was renamed prefs.HARDWARE, and now allows hardware to be configured with dictionaries rather than integers only. Initially PINS was meant to just contain pin numbering for GPIO objects, but having a single point of hardware configuration is preferable. Task.init_hardware() now respects all parameters set in prefs.

  • Throughout the code, minimal get_this type methods have begun to be replaced with @property attributes. This is because a) I love them and think they are magical, but b) will also be building Autopilot’s closed-loop infrastructure around a Qt-style signal/slot architecture that wraps @property attributes so they can be .connected to one another easily.

  • Previously it was possible to control presentation by groups of stimuli, but now it is possible to control the presentation frequency of individual stimuli.

  • PySide2 has proper support for CSS Stylesheets, so the design of Autopilot’s GUI has been marginally improved, a process that will continue in the ceaseless quest for aesthetic perfection.

  • Several setup routines have been added to make installation of opencv, pyspin, etc. easier. I also wrote a routine to download_box() files from a URL, which is mysteriously hard to do.

  • The To-Do page now reflects the full ambition of Autopilot, where before this vision was contained only in the whitepaper and a disorganized plaintext file in the repo.

  • The Subject class can now export trial data to_csv(). A very minor update, but one that is the first in a number of planned improvements to data export.

  • I have also opened up a message board in google groups to make feature requests and discuss use and development, hope to see you there :)

    https://groups.google.com/forum/#!forum/autopilot-users

New Features

  • TRANSFORMS have been introduced!!! Transform objects have a process() method that, well, transforms data in some way. Multiple transforms can be added together to make a transformation chain. This module is still very young and doesn’t have a developed API, but will be built to to automatic type compatibility checking, coersion, parallelization, and rhythm (FIFO/FILO) control. Transforms are implemented with different modalities (image, selection, logical) that imply different types of input and output data structures, but the hierarchical structure of the modules is still quite flat.

  • HARDWARE has been substantially refactored to give objects an appropriate inheritance structure. This substantially reduces effort duplication across hardware objects and makes a bunch of obvious capabilities available to all of them, for example all hardware objects are now network (init_networking()) and logging (init_logging()) capable.

    • Cameras: The cameras.Camera_CV class allows webcams/other simple cameras to be accessed through OpenCV, and the cameras.Camera_Spinnaker class allows FLIR and other cameras to be accessed through the Spinnaker SDK. Cameras are capable of encoding videos locally (with x264), streaming frames over the network, and making acquired frames available to other objects on the same computer. The Camera_Spinnaker class provides simple @property setter/getter methods for common parameters, but also makes all PySpin attributes available to the user with its get() and set() methods. The cameras.Camera metaclass is written so that new camera types can be added by overriding a few methods. A new Video_Child can be used to run a camera on a Child agent.

    • 9DOF Motion Sensor: The i2c.I2C_9DOF class can use the LSM9DS1 sensor to collect accelerometer, magnetometer, and gyroscopic data to compute unambiguous position and orientation information. We will be including calibration and computation routines that make it easier to extract properties of interest – eg. computing vertical motion by combining readings from the three sensors.

    • Temperature Sensor: The i2c.MLX90640 class can use the MLX90640 sensor to measure temperature. The sensor is 32x24px, which the class can interpolate(). The class also allows frames to be integrated and averaged over time, substantially reducing noise. I modified the driver library to enable capture at the full 64fps on the Raspberry Pi.

  • NETWORKING modules can stream continuous data better in a few ways:

    • Net_Node modules were given a get_stream() method that lets objects, well, stream data. Specifically, they are given a queue.Queue to shovel data into, which is then picked up by a dedicated zmq.Socket in its own thread, which handles batching, serialization, and load balancing. Streamed messages are batched (ie. contain multiple messages), but behave like normal message when received – they are split and contain an inner_key that is used to call the listen with each message (see l_stream()).

    • networking objects also now compress arrays-in-transit with the superfast blosc compression library. This increases their throughput dramatically, as many data streams in neuroscience are relatively low-entropy (eg. the pixels in a video of a mostly-white arena are mostly unchanged frame-to-frame and are thus highly compressible). See the Message._serialize_numpy() and Message._deserialize_numpy() methods.

  • STIMULI - The JackClient can now play continuous sounds rather than discrete sounds. An example can be found in the Nafc_Gap task, which plays continuous white noise. All sounds now have a play_continuous() method, which continually dumps samples in a cycle into a queue for the JackClient. The continuous sound will be interrupted if another sound has its Jack_Sound.play() method called, but the continuous sound will resume seamlessly even if number of samples in the played sound aren’t a multiple of the jack buffer size. We use this for gaps in noise (using the new Gap class), which we have confirmed are sample-accurate.

  • UI & VIZ

    • A Video window has been created to display streaming video. The Terminal_Networking.l_continuous() method meters frames such that even if high-speed video is being acquired, frames are only sent at a rate of prefs.DRAWFPS. The Video class uses the ImageItem_TimedUpdate object, a slight modification of pyqtgraph.ImageItem, that calls its update method according to a PySide2.QtCore.QTimer.

    • A plots_menu menu has been added to the Terminal, and a GUI dialog (gui.Psychometric) has been added to create simple psychometric curves with the viz.psychometric module, which uses altair. Plans for developing visualization are described in To-Do.

    • A general gui.pop_dialog() function simplifies displaying messages to the user using the Terminal UI. This was an initial step towards improving status/error reporting from other agents, further detailed in To-Do.

Bugfixes

  • Some objects, particularly several gui objects, had the old mouse/mice terminology updated to subject/subjects.

  • Net_Node objects were only implicitly destroyed by their release method which ends the threaded loop by setting the closing event.

  • Embarassingly, Pilot objects were not prevented from running multiple tasks at a time. This led to some very confusing and hard-to-debug problems, as well as frequent conflicts over hardware access and resources. Typically what would happen is the Terminal would send a START message to begin a task, and if it wouldn’t received a message receipt quickly enough would resend it, resulting in two tasks being started – but this would happen whenever two START messages were sent to a pilot. This was fixed with a simple check of Pilot.state before a task is initialized. Similar bugs were fixed in Plot objects.

  • The Subject class would sometimes fail to get and increment the trial session. This has been fixed by saving the session number as an attribute in the info node.

  • The Subject class would reset the session counter even when the same task was being reassigned (eg. if updated), now it preserves session number if the protocol name is unchanged.

  • The update_protocols() method didn’t report which subjects had their protocols updated, and so if there was some exception when setting new protocols it happened silently, making it so a user would never know their task was never updated. This was fixed with a noisier protocol update method for the Subject class and by displaying a list of subjects that were updated after the method is called.

  • Correction trials were being calculated incorrectly by the Stim_Manager, such that rather than only repeating a stimulus if the subject got the previous trial incorrect, the stimulus was always repeated at least once.

Code Structure

  • Modified versions of external libraries have been added as git submodules in autopilot/external.

  • Requirements files have been split out to better differentiate between different agents and use-cases. eg. requirements for Terminal agents are in requirements/requirements_terminal.txt, requirements for build the docs are in requirements/requirements_docs.txt, etc. This is a temporary arrangement, as a future design goal is restructuring setup routines so that they can flexibly install components as-needed (see To-Do)

  • autopilot.core.hardware has been refactored into its own module, autopilot.hardware, and split by device type, currently…

    • autopilot.cameras

    • autopilot.gpio - devices that use the GPIO pins for standard digital I/O logic

    • autopilot.i2c - devices that use the GPIO pins for I2C

    • autopilot.usb

  • The docs are hosted on readthedocs again, so the docs structure has been collapsed to a single folder without built documentation

  • The autopilot user directory is now ~/autopilot rather than /usr/autopilot, which was always a mistake anyway. Autopilot creates a wayfinder ~/.autopilot file that is used to find the user directory if it’s set elsewhere

External Libraries

  • External libraries can now be built and packaged along with autopilot using cmake, see CMakeLists.txt. Still uh having a little bit of trouble getting this to work, so code is in place to build and package the custom pigpio repo and jack audio but this will likely need some more work.

  • pigpio https://github.com/sneakers-the-rat/pigpio/

    • Added the ability to return absolute timestamps rather than system ticks. pigpio typically returns 1 32-bit integer of ticks since the daemon started, absolute timestamps are 64-bit, so the pigpio daemon and python interface (pi) were given two new methods:

      • synchronize gets several (default 5) sets of paired timestamps and ticks using get_sync_time. It then computes an offset for translating ticks to timestamps

      • ticks_to_timestamp converts ticks to timestamps based on the offset found with synchronize

      • get_current_time sends two requests to the daemon to get the seconds and microseconds of the complete timestamp and returns an isoformatted string

  • mlx90640-library https://github.com/pimoroni/mlx90640-library

    • Removed building examples by default which require additional dependencies

    • When using the raspi I2C driver, the baudrate would never be set to 1MHz, which is necessary to achieve full 64fps. This was fixed to use 1MHz by default.

Regressions

  • Message confirmation (holding a message to resend if confirmation isn’t received) was causing a huge amount of problems and needed to be rethought. There are in general very low rates (near-zero) of messages being dropped without some larger bug causing them, so confirmation has been disabled for now.

  • The same is true of heartbeat() - which polled for status of connected pilots. this will be repaired and restored, as the terminal currently has a pretty bad idea of the status of what’s connected to it. this will be part of a broader networking overhaul