cameras¶
Classes:
|
Metaclass for Camera objects. |
|
Capture Video from a webcam with OpenCV |
|
Capture video from a FLIR brand camera with the Spinnaker SDK. |
|
Encode frames as they are acquired in a separate process. |
Functions:
List all available Spinnaker cameras and their |
-
OPENCV_LAST_INIT_TIME
= <Synchronized wrapper for c_double(0.0)>¶ Time the last OpenCV camera was initialized (seconds, from time.time()).
v4l2 has an extraordinarily obnoxious …feature – if you try to initialize two cameras at ~the same time, you will get a neverending stream of informative error messages:
VIDIOC_QBUF: Invalid argument
The workaround seems to be relatively simple, we just wait ~2 seconds if another camera was just initialized.
-
class
Camera
(fps=None, timed=False, crop=None, **kwargs)[source]¶ Bases:
autopilot.hardware.Hardware
Metaclass for Camera objects. Should not be instantiated on its own.
- Parameters
When the camera is instantiated and
capture()
is called, the class uses a series of methods that should be overwritten in subclasses. Further details for each can be found in the relevant method documentation.It is highly recommended to instantiate Cameras with a
Hardware.name
, as it is used inoutput_filename
and to identify the network streamThree methods are required to be overwritten by all subclasses:
init_cam()
- required - used bycam
, instantiating the camera object so that it can be queried and configured_timestamp()
- required - get a timestamp for the frame
The other methods are optional and depend on the particular camera:
capture_init()
- optional - any required routine to prepare the camera after it is instantiated but before it begins to capture_process()
- optional - the wrapper around a full acquisition cycle, including streaming, writing, and queueing frames_write_frame()
- optional - how to write an individual frame to disk_write_deinit()
- optional - any required routine to finish writing to disk after acquisitioncapture_deinit()
- optional - any required routine to stop acquisition but not release the camera instance.
- Variables
frame (tuple) – The current captured frame as a tuple (timestamp, frame).
shape (tuple) – Shape of captured frames (height, width, channels)
blosc (bool) – If True (default), use blosc compression when
cam – The object used to interact with the camera
fps (int) – Framerate of video capture
timed (bool, int, float) – If False (default), camera captures indefinitely. If int or float, captures for this many seconds
q (Queue) – Queue that allows frames to be pulled by other objects
queue_size (int) – How many frames should be buffered in the queue.
initialized (threading.Event) – Called in
init_cam()
to indicate the camera has been initializedstopping (threading.Event) – Called to signal that capturing should stop. when set, ends the threaded capture loop
capturing (threading.Event) – Set when camera is actively capturing
streaming (threading.Event) – Set to indicate that the camera is streaming data over the network
writing (threading.Event) – Set to indicate that the camera is writing video locally
queueing (threading.Event) – Indicates whether frames are being put into
q
indicating (threading.Event) – Set to indicate that capture progress is being indicated in stdout by
tqdm
- Parameters
fps
timed
crop (tuple) – (x, y of top left corner, width, height)
**kwargs
Attributes:
test documenting input
what are we anyway?
Camera object.
Filename given to video writer.
Methods:
capture
([timed])Spawn a thread to begin capturing.
_capture
()Threaded capture method started by
capture()
._process
()A full frame capture cycle.
stream
([to, ip, port, min_size])Enable streaming frames on capture.
l_start
(val)Begin capturing by calling
Camera.capture()
l_stop
(val)Stop capture by calling
Camera.release()
write
([output_filename, timestamps, blosc])Enable writing frames locally on capture
Put
frame
into the_write_q
, optionally compressing it withblosc.pack_array()
End the
Video_Writer
.queue
([queue_size])Enable stashing frames in a queue for a local consumer.
_grab
()Capture a frame and timestamp.
_timestamp
([frame])Generate a timestamp for each
_grab()
init_cam
()Method to initialize camera object
Optional: Prepare
cam
after initialization, but before captureOptional: Return
cam
to an idle state after capturing, but before releasingstop
()Stop capture by setting
stopping
release
()Release resources held by Camera.
-
input
= True¶ test documenting input
-
capture
(timed=None)[source]¶ Spawn a thread to begin capturing.
- Parameters
timed (None, int, float) – if None, record according to
timed
(default). If numeric, record fortimed
seconds.
-
_capture
()[source]¶ Threaded capture method started by
capture()
.Captures until
stopping
is set.Calls capture methods, in order:
capture_init()
- any required routine to prepare the camera after it is instantiated but before it begins to capture_process()
- the wrapper around a full acquisition cycle, including streaming, writing, and queueing frames_timestamp()
- get a timestamp for the frame_write_frame()
- how to write an individual frame to disk_write_deinit()
- any required routine to finish writing to disk after acquisitioncapture_deinit()
- any required routine to stop acquisition but not release the camera instance.
Returns:
-
_process
()[source]¶ A full frame capture cycle.
_grab`s the :attr:()
.frame`, then handles streaming, writing, queueing, and indicating according tostream()
,write()
,queue()
, andindicating
, respectively.
-
stream
(to='T', ip=None, port=None, min_size=5, **kwargs)[source]¶ Enable streaming frames on capture.
Spawns a
Net_Node
withHardware.init_networking()
, and creates a streaming queue withNet_Node.get_stream()
according to args.Sets
Camera.streaming
- Parameters
to (str) – ID of the recipient. Default ‘T’ for Terminal.
ip (str) – IP of recipient. If None (default), ‘localhost’. If None and
to
is ‘T’,prefs.TERMINALIP
port (int, str) – Port of recipient socket. If None (default),
prefs.MSGPORT
. If None andto
is ‘T’,prefs.TERMINALPORT
.**kwargs – passed to
Hardware.init_networking()
and thus toNet_Node
-
l_start
(val)[source]¶ Begin capturing by calling
Camera.capture()
- Parameters
val – unused
-
l_stop
(val)[source]¶ Stop capture by calling
Camera.release()
- Parameters
val – unused
-
write
(output_filename=None, timestamps=True, blosc=True)[source]¶ Enable writing frames locally on capture
Spawns a
Video_Writer
to encode video, setswriting
- Parameters
output_filename (str) – path and filename of the output video. extension should be
.mp4
, as videos are encoded with libx264 by default.timestamps (bool) – if True, (timestamp, frame) tuples will be put in the
_write_q
. if False, timestamps will be generated byVideo_Writer
(not recommended at all).blosc (bool) – if true, compress frames with
blosc.pack_array()
before putting in_write_q
.
-
_write_frame
()[source]¶ Put
frame
into the_write_q
, optionally compressing it withblosc.pack_array()
-
_write_deinit
()[source]¶ End the
Video_Writer
.Blocks until the
_write_q
is empty, holding the release of the object.
-
queue
(queue_size=128)[source]¶ Enable stashing frames in a queue for a local consumer.
Other objects can get frames as they are acquired from
q
- Parameters
queue_size (int) – max number of frames that can be held in
q
-
property
cam
¶ Camera object.
If
_cam
hasn’t been initialized yet, useinit_cam()
to do so- Returns
Camera object, different for each camera.
-
property
output_filename
¶ Filename given to video writer.
If explicitly set, returns as expected.
If None, or path already exists while the camera isn’t capturing, a new filename is generated in the user directory.
- Returns
(str)
_output_filename
-
_grab
()[source]¶ Capture a frame and timestamp.
Method must be overridden by subclass
- Returns
- (str,
numpy.ndarray
) Tuple of isoformatted (str) or numeric timestamp returned by_timestamp()
, and captured frame
- (str,
-
_timestamp
(frame=None)[source]¶ Generate a timestamp for each
_grab()
Must be overridden by subclass
- Parameters
frame – If needed by camera subclass, pass the frame or image object to get timestamp
- Returns
(str, int, float) Either an isoformatted (str) or numeric timestamp
-
init_cam
()[source]¶ Method to initialize camera object
Must be overridden by camera subclass
- Returns
camera object
-
class
Camera_CV
(camera_idx=0, **kwargs)[source]¶ Bases:
autopilot.hardware.cameras.Camera
Capture Video from a webcam with OpenCV
By default, OpenCV will select a suitable backend for the indicated camera. Some backends have difficulty operating multiple cameras at once, so the performance of this class will be variable depending on camera type.
- Parameters
camera_idx (int) – The index of the desired camera
**kwargs – Passed to the
Camera
metaclass.
- Variables
camera_idx (int) – The index of the desired camera
last_opencv_init (float) – See
OPENCV_LAST_INIT_TIME
last_init_lock (
threading.Lock
) – Lock for settinglast_opencv_init
Note
OpenCV must be installed to use this class! A Prebuilt opencv binary is available for the raspberry pi, but it doesn’t take advantage of some performance-enhancements available to OpenCV. Use the
install_opencv.sh
script in the setup directory to compile OpenCV with these enhancements.If your camera isn’t working, to print debugging information you can run:
echo 3 > /sys/class/video4linux/videox/dev_debug # check logs dmesg
Attributes:
Attempts to get FPS with
cv2.CAP_PROP_FPS
, uses 30fps as a defaultAttempts to get image shape from
cv2.CAP_PROP_FRAME_WIDTH
andHEIGHT
:returns: (width, height) :rtype: tuplecapture backend used by OpenCV for this camera
Device information from
v4l2-ctl
Methods:
_grab
()Reads a frame with
cam.read()
_timestamp
([frame])Attempts to get timestamp with
cv2.CAP_PROP_POS_MSEC
.init_cam
()Initializes OpenCV Camera
release
()Release resources held by Camera.
-
property
fps
¶ Attempts to get FPS with
cv2.CAP_PROP_FPS
, uses 30fps as a default- Returns
framerate
- Return type
-
property
shape
¶ Attempts to get image shape from
cv2.CAP_PROP_FRAME_WIDTH
andHEIGHT
:returns: (width, height) :rtype: tuple
-
_timestamp
(frame=None)[source]¶ Attempts to get timestamp with
cv2.CAP_PROP_POS_MSEC
. Frame does not need to be passed to this method, as timestamps are retrieved fromcam
Todo
Convert this float timestamp to an isoformatted system timestamp
- Returns
milliseconds since capture start
- Return type
-
property
backend
¶ capture backend used by OpenCV for this camera
- Returns
name of capture backend used by OpenCV for this camera
- Return type
-
init_cam
()[source]¶ Initializes OpenCV Camera
To avoid overlapping resource allocation requests, checks the last time any
Camera_CV
object was instantiated and makes sure it has been at least 2 seconds since then.- Returns
camera object
- Return type
cv2.VideoCapture
-
class
Camera_Spinnaker
(serial=None, camera_idx=None, **kwargs)[source]¶ Bases:
autopilot.hardware.cameras.Camera
Capture video from a FLIR brand camera with the Spinnaker SDK.
- Parameters
serial (str) – Serial number of desired camera
camera_idx (int) – If no serial provided, select camera by index. Using
serial
is HIGHLY RECOMMENDED.**kwargs – passed to
Camera
metaclass
Note
PySpin and the Spinnaker SDK must be installed to use this class. Please use the
install_pyspin.sh
script insetup
See the documentation for the Spinnaker SDK and PySpin here:
https://www.flir.com/products/spinnaker-sdk/
- Variables
serial (str) – Serial number of desired camera
camera_idx (int) – If no serial provided, select camera by index. Using
serial
is HIGHLY RECOMMENDED.system (
PySpin.System
) – The PySpin System objectcam_list (
PySpin.CameraList
) – The list of PySpin Cameras available to the systemnmap – A reference to the nodemap from the GenICam XML description of the device
base_path (str) –
The directory and base filename that images will be written to if object is
writing
. eg:base_path = ‘/home/user/capture_directory/capture_’ image_path = base_path + ‘image1.png’
img_opts (
PySpin.PNGOption
) – Options for saving .png images, made bywrite()
Attributes:
Conversion from data types to pointer types
Conversion from data types to human-readable names
bool, ‘write’:bool} descriptor
Camera Binning.
Set Exposure of camera
Acquisition Framerate
Set camera to lead or follow hardware triggers
Image acquisition mode
All device attributes that are currently readable with
get()
All device attributes that are currently writeable wth
set()
Get all information about the camera
Methods:
init_cam
()Initialize the Spinnaker Camera
Prepare the camera for acquisition
De-initializes the camera after acquisition
_process
()Modification of the
Camera._process()
method for Spinnaker cameras_grab
()Get next timestamp and PySpin Image
_timestamp
([frame])Get the timestamp from the passed image
write
([output_filename, timestamps, blosc])Sets camera to save acquired images to a directory for later encoding.
Write frame to
base_path
+ timestamp + ‘.png’ withPySpin.Image.Save()
After capture, write images in
base_path
to video withDirectory_Writer
get
(attr)Get a camera attribute.
set
(attr, val)Set a camera attribute
list_options
(name)List the possible values of a camera attribute.
release
()Release all PySpin objects and wait on writer, if still active.
-
ATTR_TYPES
= {}¶ Conversion from data types to pointer types
-
ATTR_TYPE_NAMES
= {}¶ Conversion from data types to human-readable names
-
RW_MODES
= {}¶ bool, ‘write’:bool} descriptor
- Type
Conversion from read/write mode to {‘read’
-
init_cam
()[source]¶ Initialize the Spinnaker Camera
Initializes the camera, system, cam_list, node map, and the camera methods and attributes used by
get()
andset()
- Returns
The Spinnaker camera object
- Return type
PySpin.Camera
-
capture_init
()[source]¶ Prepare the camera for acquisition
calls the camera’s
BeginAcquisition
method and populateshape
-
_process
()[source]¶ Modification of the
Camera._process()
method for Spinnaker camerasBecause the objects returned from the
_grab()
method are image pointers rather than :class:`numpy.ndarray`s, they need to be handled differently.More details on the differences are given in the
_write_frame()
,
-
_timestamp
(frame=None)[source]¶ Get the timestamp from the passed image
- Parameters
frame (
PySpin.Image
) – Currently grabbed image- Returns
PySpin timestamp
- Return type
-
write
(output_filename=None, timestamps=True, blosc=True)[source]¶ Sets camera to save acquired images to a directory for later encoding.
For performance, rather than encoding during acquisition, save each image as a (lossless) .png image in a directory generated by
output_filename
.After capturing is complete, a
Directory_Writer
encodes the images to an x264 encoded .mp4 video.- Parameters
output_filename (str) – Directory to write images to. If None (default), generated by
output_filename
timestamps (bool) – Not used, timestamps are always appended to filenames.
blosc (bool) – Not used, images are directly saved.
-
_write_deinit
()[source]¶ After capture, write images in
base_path
to video withDirectory_Writer
Camera object will remain open until writer has finished.
-
property
bin
¶ Camera Binning.
Attempts to bin on-device, and use averaging if possible. If averaging not available, uses summation.
- Parameters
tuple – tuple of integers, (Horizontal, Vertical binning)
- Returns
(Horizontal, Vertical binning)
- Return type
-
property
exposure
¶ Set Exposure of camera
Can be set with
'auto'
- automatic exposure control. note that this will limit frameratefloat
from 0-1 - exposure duration proportional to fps. eg. if fps = 10, setting exposure = 0.5 means exposure will be set as 50msfloat
orint
>1 - absolute exposure time in microseconds
-
property
fps
¶ Acquisition Framerate
Set with integer. If set with None, ignored (superclass sets FPS to None on init)
- Returns
from
cam.AcquisitionFrameRate.GetValue()
- Return type
-
property
frame_trigger
¶ Set camera to lead or follow hardware triggers
If
'lead'
, Camera will send TTL pulses from Line 2.If
'follow'
, Camera will follow triggers from Line 3.
-
property
acquisition_mode
¶ Image acquisition mode
One of
'continuous'
- continuously acquire frame camera'single'
- acquire a single frame'multi'
- acquire a finite number of frames.
Warning
Only
'continuous'
has been tested.
-
property
readable_attributes
¶ All device attributes that are currently readable with
get()
- Returns
A dictionary of attributes that are readable and their current values
- Return type
-
property
writable_attributes
¶ All device attributes that are currently writeable wth
set()
- Returns
A dictionary of attributes that are writeable and their current values
- Return type
-
get
(attr)[source]¶ Get a camera attribute.
Any value in
readable_attributes
can be read. Attempts to get numeric values with.GetValue
, otherwise gets a string with.ToString
, so be cautious with types.If
attr
is a method (ie. in._camera_methods
, execute the method and return the value
-
set
(attr, val)[source]¶ Set a camera attribute
Any value in
writeable_attributes
can be set. If attribute has a.SetValue
method, (ie. accepts numeric values), attempt to use it, otherwise use.FromString
.- Parameters
attr (str) – Name of attribute to be set
val (str, int, float) – Value to set attribute
-
list_options
(name)[source]¶ List the possible values of a camera attribute.
- Parameters
name (str) – name of attribute to query
- Returns
Dictionary with {available options: descriptions}
- Return type
-
property
device_info
¶ Get all information about the camera
Note that this is distinct from camera attributes like fps, instead this is information like serial number, version, firmware revision, etc.
- Returns
{feature name: feature value}
- Return type
-
class
Video_Writer
(q, path, fps=None, timestamps=True, blosc=True)[source]¶ Bases:
multiprocessing.context.Process
Encode frames as they are acquired in a separate process.
Must call
start()
after initialization to begin encoding.Encoding continues until ‘END’ is put in
q
.Timestamps are saved in a .csv file with the same path as the video.
- Parameters
q (
Queue
) – Queue into which frames will be dumpedpath (str) – output path of video
fps (int) – framerate of output video
timestamps (bool) – if True (default), input will be of form (timestamp, frame). if False, input will just be frames and timestamps will be generated as the frame is encoded (not recommended)
blosc (bool) – if True, frames in the
q
will be compresed with blosc. if False, uncompressed
- Variables
timestamps (list) – Timestamps for frames, written to .csv on completion of encoding
Methods:
run
()Open a
skvideo.io.FFmpegWriter
and begin processing frames fromq
-
run
()[source]¶ Open a
skvideo.io.FFmpegWriter
and begin processing frames fromq
Should not be called by itself, overwrites the
multiprocessing.Process.run()
method, so should callVideo_Writer.start()
Continue encoding until ‘END’ put in queue.