cameras

Inheritance diagram of autopilot.hardware.cameras

Classes:

Camera([fps, timed, crop])

Metaclass for Camera objects.

Camera_CV([camera_idx])

Capture Video from a webcam with OpenCV

Camera_Spinnaker([serial, camera_idx])

Capture video from a FLIR brand camera with the Spinnaker SDK.

Video_Writer(q, path[, fps, timestamps, blosc])

Encode frames as they are acquired in a separate process.

Functions:

list_spinnaker_cameras()

List all available Spinnaker cameras and their DeviceInformation

OPENCV_LAST_INIT_TIME = <Synchronized wrapper for c_double(0.0)>

Time the last OpenCV camera was initialized (seconds, from time.time()).

v4l2 has an extraordinarily obnoxious …feature – if you try to initialize two cameras at ~the same time, you will get a neverending stream of informative error messages: VIDIOC_QBUF: Invalid argument

The workaround seems to be relatively simple, we just wait ~2 seconds if another camera was just initialized.

class Camera(fps=None, timed=False, crop=None, **kwargs)[source]

Bases: autopilot.hardware.Hardware

Metaclass for Camera objects. Should not be instantiated on its own.

Parameters
  • fps (int) – Framerate of video capture

  • timed (bool, int, float) – If False (default), camera captures indefinitely. If int or float, captures for this many seconds

  • **kwargs

    Arguments to stream(), write(), and queue() can be passed as dictionaries, eg.:

    stream={'to':'T', 'ip':'localhost'}
    

When the camera is instantiated and capture() is called, the class uses a series of methods that should be overwritten in subclasses. Further details for each can be found in the relevant method documentation.

It is highly recommended to instantiate Cameras with a Hardware.name, as it is used in output_filename and to identify the network stream

Three methods are required to be overwritten by all subclasses:

  • init_cam() - required - used by cam, instantiating the camera object so that it can be queried and configured

  • _grab() - required - grab a frame from the cam

  • _timestamp() - required - get a timestamp for the frame

The other methods are optional and depend on the particular camera:

  • capture_init() - optional - any required routine to prepare the camera after it is instantiated but before it begins to capture

  • _process() - optional - the wrapper around a full acquisition cycle, including streaming, writing, and queueing frames

  • _write_frame() - optional - how to write an individual frame to disk

  • _write_deinit() - optional - any required routine to finish writing to disk after acquisition

  • capture_deinit() - optional - any required routine to stop acquisition but not release the camera instance.

Variables
  • frame (tuple) – The current captured frame as a tuple (timestamp, frame).

  • shape (tuple) – Shape of captured frames (height, width, channels)

  • blosc (bool) – If True (default), use blosc compression when

  • cam – The object used to interact with the camera

  • fps (int) – Framerate of video capture

  • timed (bool, int, float) – If False (default), camera captures indefinitely. If int or float, captures for this many seconds

  • q (Queue) – Queue that allows frames to be pulled by other objects

  • queue_size (int) – How many frames should be buffered in the queue.

  • initialized (threading.Event) – Called in init_cam() to indicate the camera has been initialized

  • stopping (threading.Event) – Called to signal that capturing should stop. when set, ends the threaded capture loop

  • capturing (threading.Event) – Set when camera is actively capturing

  • streaming (threading.Event) – Set to indicate that the camera is streaming data over the network

  • writing (threading.Event) – Set to indicate that the camera is writing video locally

  • queueing (threading.Event) – Indicates whether frames are being put into q

  • indicating (threading.Event) – Set to indicate that capture progress is being indicated in stdout by tqdm

Parameters
  • fps

  • timed

  • crop (tuple) – (x, y of top left corner, width, height)

  • **kwargs

Attributes:

input

test documenting input

type

what are we anyway?

cam

Camera object.

output_filename

Filename given to video writer.

Methods:

capture([timed])

Spawn a thread to begin capturing.

_capture()

Threaded capture method started by capture().

_process()

A full frame capture cycle.

stream([to, ip, port, min_size])

Enable streaming frames on capture.

l_start(val)

Begin capturing by calling Camera.capture()

l_stop(val)

Stop capture by calling Camera.release()

write([output_filename, timestamps, blosc])

Enable writing frames locally on capture

_write_frame()

Put frame into the _write_q, optionally compressing it with blosc.pack_array()

_write_deinit()

End the Video_Writer.

queue([queue_size])

Enable stashing frames in a queue for a local consumer.

_grab()

Capture a frame and timestamp.

_timestamp([frame])

Generate a timestamp for each _grab()

init_cam()

Method to initialize camera object

capture_init()

Optional: Prepare cam after initialization, but before capture

capture_deinit()

Optional: Return cam to an idle state after capturing, but before releasing

stop()

Stop capture by setting stopping

release()

Release resources held by Camera.

input = True

test documenting input

type = 'CAMERA'

what are we anyway?

Type

(str)

capture(timed=None)[source]

Spawn a thread to begin capturing.

Parameters

timed (None, int, float) – if None, record according to timed (default). If numeric, record for timed seconds.

_capture()[source]

Threaded capture method started by capture().

Captures until stopping is set.

Calls capture methods, in order:

  • capture_init() - any required routine to prepare the camera after it is instantiated but before it begins to capture

  • _process() - the wrapper around a full acquisition cycle, including streaming, writing, and queueing frames

  • _grab() - grab a frame from the cam

  • _timestamp() - get a timestamp for the frame

  • _write_frame() - how to write an individual frame to disk

  • _write_deinit() - any required routine to finish writing to disk after acquisition

  • capture_deinit() - any required routine to stop acquisition but not release the camera instance.

Returns:

_process()[source]

A full frame capture cycle.

_grab`s the :attr:().frame`, then handles streaming, writing, queueing, and indicating according to stream(), write(), queue(), and indicating, respectively.

stream(to='T', ip=None, port=None, min_size=5, **kwargs)[source]

Enable streaming frames on capture.

Spawns a Net_Node with Hardware.init_networking(), and creates a streaming queue with Net_Node.get_stream() according to args.

Sets Camera.streaming

Parameters
  • to (str) – ID of the recipient. Default ‘T’ for Terminal.

  • ip (str) – IP of recipient. If None (default), ‘localhost’. If None and to is ‘T’, prefs.TERMINALIP

  • port (int, str) – Port of recipient socket. If None (default), prefs.MSGPORT. If None and to is ‘T’, prefs.TERMINALPORT.

  • **kwargs – passed to Hardware.init_networking() and thus to Net_Node

l_start(val)[source]

Begin capturing by calling Camera.capture()

Parameters

val – unused

l_stop(val)[source]

Stop capture by calling Camera.release()

Parameters

val – unused

write(output_filename=None, timestamps=True, blosc=True)[source]

Enable writing frames locally on capture

Spawns a Video_Writer to encode video, sets writing

Parameters
  • output_filename (str) – path and filename of the output video. extension should be .mp4, as videos are encoded with libx264 by default.

  • timestamps (bool) – if True, (timestamp, frame) tuples will be put in the _write_q. if False, timestamps will be generated by Video_Writer (not recommended at all).

  • blosc (bool) – if true, compress frames with blosc.pack_array() before putting in _write_q.

_write_frame()[source]

Put frame into the _write_q, optionally compressing it with blosc.pack_array()

_write_deinit()[source]

End the Video_Writer.

Blocks until the _write_q is empty, holding the release of the object.

queue(queue_size=128)[source]

Enable stashing frames in a queue for a local consumer.

Other objects can get frames as they are acquired from q

Parameters

queue_size (int) – max number of frames that can be held in q

property cam

Camera object.

If _cam hasn’t been initialized yet, use init_cam() to do so

Returns

Camera object, different for each camera.

property output_filename

Filename given to video writer.

If explicitly set, returns as expected.

If None, or path already exists while the camera isn’t capturing, a new filename is generated in the user directory.

Returns

(str) _output_filename

_grab()[source]

Capture a frame and timestamp.

Method must be overridden by subclass

Returns

(str, numpy.ndarray) Tuple of isoformatted (str) or numeric timestamp returned by _timestamp(),

and captured frame

_timestamp(frame=None)[source]

Generate a timestamp for each _grab()

Must be overridden by subclass

Parameters

frame – If needed by camera subclass, pass the frame or image object to get timestamp

Returns

(str, int, float) Either an isoformatted (str) or numeric timestamp

init_cam()[source]

Method to initialize camera object

Must be overridden by camera subclass

Returns

camera object

capture_init()[source]

Optional: Prepare cam after initialization, but before capture

Returns

None

capture_deinit()[source]

Optional: Return cam to an idle state after capturing, but before releasing

Returns

None

stop()[source]

Stop capture by setting stopping

release()[source]

Release resources held by Camera.

Must be overridden by subclass.

Does not raise exception in case some general camera release logic should be put here…

class Camera_CV(camera_idx=0, **kwargs)[source]

Bases: autopilot.hardware.cameras.Camera

Capture Video from a webcam with OpenCV

By default, OpenCV will select a suitable backend for the indicated camera. Some backends have difficulty operating multiple cameras at once, so the performance of this class will be variable depending on camera type.

Parameters
  • camera_idx (int) – The index of the desired camera

  • **kwargs – Passed to the Camera metaclass.

Variables
  • camera_idx (int) – The index of the desired camera

  • last_opencv_init (float) – See OPENCV_LAST_INIT_TIME

  • last_init_lock (threading.Lock) – Lock for setting last_opencv_init

Note

OpenCV must be installed to use this class! A Prebuilt opencv binary is available for the raspberry pi, but it doesn’t take advantage of some performance-enhancements available to OpenCV. Use the install_opencv.sh script in the setup directory to compile OpenCV with these enhancements.

If your camera isn’t working, to print debugging information you can run:

echo 3 > /sys/class/video4linux/videox/dev_debug

# check logs
dmesg

Attributes:

fps

Attempts to get FPS with cv2.CAP_PROP_FPS, uses 30fps as a default

shape

Attempts to get image shape from cv2.CAP_PROP_FRAME_WIDTH and HEIGHT :returns: (width, height) :rtype: tuple

backend

capture backend used by OpenCV for this camera

v4l_info

Device information from v4l2-ctl

Methods:

_grab()

Reads a frame with cam.read()

_timestamp([frame])

Attempts to get timestamp with cv2.CAP_PROP_POS_MSEC.

init_cam()

Initializes OpenCV Camera

release()

Release resources held by Camera.

property fps

Attempts to get FPS with cv2.CAP_PROP_FPS, uses 30fps as a default

Returns

framerate

Return type

int

property shape

Attempts to get image shape from cv2.CAP_PROP_FRAME_WIDTH and HEIGHT :returns: (width, height) :rtype: tuple

_grab()[source]

Reads a frame with cam.read()

Returns

(timestamp, frame)

Return type

tuple

_timestamp(frame=None)[source]

Attempts to get timestamp with cv2.CAP_PROP_POS_MSEC. Frame does not need to be passed to this method, as timestamps are retrieved from cam

Todo

Convert this float timestamp to an isoformatted system timestamp

Returns

milliseconds since capture start

Return type

float

property backend

capture backend used by OpenCV for this camera

Returns

name of capture backend used by OpenCV for this camera

Return type

str

init_cam()[source]

Initializes OpenCV Camera

To avoid overlapping resource allocation requests, checks the last time any Camera_CV object was instantiated and makes sure it has been at least 2 seconds since then.

Returns

camera object

Return type

cv2.VideoCapture

release()[source]

Release resources held by Camera.

Must be overridden by subclass.

Does not raise exception in case some general camera release logic should be put here…

property v4l_info

Device information from v4l2-ctl

Returns

Information for all devices available through v4l2

Return type

dict

class Camera_Spinnaker(serial=None, camera_idx=None, **kwargs)[source]

Bases: autopilot.hardware.cameras.Camera

Capture video from a FLIR brand camera with the Spinnaker SDK.

Parameters
  • serial (str) – Serial number of desired camera

  • camera_idx (int) – If no serial provided, select camera by index. Using serial is HIGHLY RECOMMENDED.

  • **kwargs – passed to Camera metaclass

Note

PySpin and the Spinnaker SDK must be installed to use this class. Please use the install_pyspin.sh script in setup

See the documentation for the Spinnaker SDK and PySpin here:

https://www.flir.com/products/spinnaker-sdk/

Variables
  • serial (str) – Serial number of desired camera

  • camera_idx (int) – If no serial provided, select camera by index. Using serial is HIGHLY RECOMMENDED.

  • system (PySpin.System) – The PySpin System object

  • cam_list (PySpin.CameraList) – The list of PySpin Cameras available to the system

  • nmap – A reference to the nodemap from the GenICam XML description of the device

  • base_path (str) –

    The directory and base filename that images will be written to if object is writing. eg:

    base_path = ‘/home/user/capture_directory/capture_’ image_path = base_path + ‘image1.png’

  • img_opts (PySpin.PNGOption) – Options for saving .png images, made by write()

Attributes:

ATTR_TYPES

Conversion from data types to pointer types

ATTR_TYPE_NAMES

Conversion from data types to human-readable names

RW_MODES

bool, ‘write’:bool} descriptor

bin

Camera Binning.

exposure

Set Exposure of camera

fps

Acquisition Framerate

frame_trigger

Set camera to lead or follow hardware triggers

acquisition_mode

Image acquisition mode

readable_attributes

All device attributes that are currently readable with get()

writable_attributes

All device attributes that are currently writeable wth set()

device_info

Get all information about the camera

Methods:

init_cam()

Initialize the Spinnaker Camera

capture_init()

Prepare the camera for acquisition

capture_deinit()

De-initializes the camera after acquisition

_process()

Modification of the Camera._process() method for Spinnaker cameras

_grab()

Get next timestamp and PySpin Image

_timestamp([frame])

Get the timestamp from the passed image

write([output_filename, timestamps, blosc])

Sets camera to save acquired images to a directory for later encoding.

_write_frame()

Write frame to base_path + timestamp + ‘.png’ with PySpin.Image.Save()

_write_deinit()

After capture, write images in base_path to video with Directory_Writer

get(attr)

Get a camera attribute.

set(attr, val)

Set a camera attribute

list_options(name)

List the possible values of a camera attribute.

release()

Release all PySpin objects and wait on writer, if still active.

ATTR_TYPES = {}

Conversion from data types to pointer types

ATTR_TYPE_NAMES = {}

Conversion from data types to human-readable names

RW_MODES = {}

bool, ‘write’:bool} descriptor

Type

Conversion from read/write mode to {‘read’

init_cam()[source]

Initialize the Spinnaker Camera

Initializes the camera, system, cam_list, node map, and the camera methods and attributes used by get() and set()

Returns

The Spinnaker camera object

Return type

PySpin.Camera

capture_init()[source]

Prepare the camera for acquisition

calls the camera’s BeginAcquisition method and populate shape

capture_deinit()[source]

De-initializes the camera after acquisition

_process()[source]

Modification of the Camera._process() method for Spinnaker cameras

Because the objects returned from the _grab() method are image pointers rather than :class:`numpy.ndarray`s, they need to be handled differently.

More details on the differences are given in the _write_frame(),

_grab()[source]

Get next timestamp and PySpin Image

Returns

(timestamp, PySpin.Image)

Return type

tuple

_timestamp(frame=None)[source]

Get the timestamp from the passed image

Parameters

frame (PySpin.Image) – Currently grabbed image

Returns

PySpin timestamp

Return type

float

write(output_filename=None, timestamps=True, blosc=True)[source]

Sets camera to save acquired images to a directory for later encoding.

For performance, rather than encoding during acquisition, save each image as a (lossless) .png image in a directory generated by output_filename.

After capturing is complete, a Directory_Writer encodes the images to an x264 encoded .mp4 video.

Parameters
  • output_filename (str) – Directory to write images to. If None (default), generated by output_filename

  • timestamps (bool) – Not used, timestamps are always appended to filenames.

  • blosc (bool) – Not used, images are directly saved.

_write_frame()[source]

Write frame to base_path + timestamp + ‘.png’ with PySpin.Image.Save()

_write_deinit()[source]

After capture, write images in base_path to video with Directory_Writer

Camera object will remain open until writer has finished.

property bin

Camera Binning.

Attempts to bin on-device, and use averaging if possible. If averaging not available, uses summation.

Parameters

tuple – tuple of integers, (Horizontal, Vertical binning)

Returns

(Horizontal, Vertical binning)

Return type

tuple

property exposure

Set Exposure of camera

Can be set with

  • 'auto' - automatic exposure control. note that this will limit framerate

  • float from 0-1 - exposure duration proportional to fps. eg. if fps = 10, setting exposure = 0.5 means exposure will be set as 50ms

  • float or int >1 - absolute exposure time in microseconds

Returns

If exposure has been set, return set value. Otherwise return .get('ExposureTime')

Return type

str, float

property fps

Acquisition Framerate

Set with integer. If set with None, ignored (superclass sets FPS to None on init)

Returns

from cam.AcquisitionFrameRate.GetValue()

Return type

int

property frame_trigger

Set camera to lead or follow hardware triggers

If 'lead', Camera will send TTL pulses from Line 2.

If 'follow', Camera will follow triggers from Line 3.

property acquisition_mode

Image acquisition mode

One of

  • 'continuous' - continuously acquire frame camera

  • 'single' - acquire a single frame

  • 'multi' - acquire a finite number of frames.

Warning

Only 'continuous' has been tested.

property readable_attributes

All device attributes that are currently readable with get()

Returns

A dictionary of attributes that are readable and their current values

Return type

dict

property writable_attributes

All device attributes that are currently writeable wth set()

Returns

A dictionary of attributes that are writeable and their current values

Return type

dict

get(attr)[source]

Get a camera attribute.

Any value in readable_attributes can be read. Attempts to get numeric values with .GetValue, otherwise gets a string with .ToString, so be cautious with types.

If attr is a method (ie. in ._camera_methods, execute the method and return the value

Parameters

attr (str) – Name of a readable attribute or executable method

Returns

Value of attr

Return type

float, int, str

set(attr, val)[source]

Set a camera attribute

Any value in writeable_attributes can be set. If attribute has a .SetValue method, (ie. accepts numeric values), attempt to use it, otherwise use .FromString.

Parameters
  • attr (str) – Name of attribute to be set

  • val (str, int, float) – Value to set attribute

list_options(name)[source]

List the possible values of a camera attribute.

Parameters

name (str) – name of attribute to query

Returns

Dictionary with {available options: descriptions}

Return type

dict

property device_info

Get all information about the camera

Note that this is distinct from camera attributes like fps, instead this is information like serial number, version, firmware revision, etc.

Returns

{feature name: feature value}

Return type

dict

release()[source]

Release all PySpin objects and wait on writer, if still active.

class Video_Writer(q, path, fps=None, timestamps=True, blosc=True)[source]

Bases: multiprocessing.context.Process

Encode frames as they are acquired in a separate process.

Must call start() after initialization to begin encoding.

Encoding continues until ‘END’ is put in q.

Timestamps are saved in a .csv file with the same path as the video.

Parameters
  • q (Queue) – Queue into which frames will be dumped

  • path (str) – output path of video

  • fps (int) – framerate of output video

  • timestamps (bool) – if True (default), input will be of form (timestamp, frame). if False, input will just be frames and timestamps will be generated as the frame is encoded (not recommended)

  • blosc (bool) – if True, frames in the q will be compresed with blosc. if False, uncompressed

Variables

timestamps (list) – Timestamps for frames, written to .csv on completion of encoding

Methods:

run()

Open a skvideo.io.FFmpegWriter and begin processing frames from q

run()[source]

Open a skvideo.io.FFmpegWriter and begin processing frames from q

Should not be called by itself, overwrites the multiprocessing.Process.run() method, so should call Video_Writer.start()

Continue encoding until ‘END’ put in queue.

list_spinnaker_cameras()[source]

List all available Spinnaker cameras and their DeviceInformation

Returns

list of dictionaries of device information for each camera.

Return type

list