An object which owns some sort of input video stream, manages running it (and any tracking, etc) and emits cv::Mat frames and tracked objects. More...

#include <TrackingStream.hpp>

Inheritance diagram for TrackingStream:

[legend]

Collaboration diagram for TrackingStream:

[legend]

Public Slots
void	start (int cam=0)
	Start a stream with a camera input.

void	start (QString fname)
	Start a stream with a filename input.

void	stop ()
	Terminate the video stream.

void	pauseStream ()
	Pause the current video stream.

void	changeProjectorCorners (std::vector< cv::Point2f > corners)
	Set the location of projector corners in the image.

void	changeClipTrack (bool checked)
	Set whether or not tracking is clipped to projector.

void	predictionBall (KFPrediction pred)
	Alert the stream to a predicted ball.

void	setFov (int fov)
	Set the camera FOV.

void	changeProjWidth (int width)
	changeProjWidth sets the width of the projector screen (in inches) More...

void	changeProjHeight (int height)
	changeProjHeight sets the height of the projector screen (in inches) More...

void	changeBallZ (double z)
	changeBallZ sets the z-plane that the ball is thrown in (in inches) More...

void	changeBlurSize (double blurSize)
	Set the strength of blur in preprocessing step. More...

void	changeThreshVal (double thresh)
	Set the threshold in preprocessing step. More...

void	changeMinRadius (double radius)
	Set the min viable radius of a ball blob. More...

void	changeMaxRadius (double radius)
	Set the max viable radius of a ball blob. More...

void	changeGravConstant (double g)
	Set the accel due to gravity g. More...

void	changeXYCovariance (double sigma)
	Set the covariance in predicted X, Y (a KF parameter) More...

void	processThresh (const cv::Mat &, enum ColorSpace=TS_GRAY)
	Process a "thresholded" frame, with white blobs.

void	processBlur (const cv::Mat &, enum ColorSpace=TS_BGR)
	Process a "blurred" frame.

void	processContour (const cv::Mat &, enum ColorSpace=TS_GRAY)
	Process a "contour" frame, which marks contours.

void	processVideoFrame (const cv::Mat &mat, ColorSpace=TS_BGR)
	Process a "video" frame, which is raw video input.

void	changeDisplayFrameType (QString s)
	Set the type of video frame to actually be emitted.

Signals
void	frameReady (const cv::Mat &, enum ColorSpace=TS_BGR)
	frameReady emits a cv::Mat of specified color space for processing

void	ballSpotted (TrackingBall ball)
	A ball has been spotted (image coordinates)

void	ballPredicted (KFPrediction pred)
	A ball has been predicted (image coordinates)

void	ballProjSpotted (TrackingBall ball)
	A ball has been spotted (proj coordinates)

void	ballProjPredicted (KFPrediction pred)
	A ball has been predicted (proj coordinates)

void	ballLost ()
	We have lost track of the ball entirely.

void	started ()
	Signal emitted when stream starts processing.

Public Member Functions
	TrackingStream (QObject *parent=0)

void	startProcess ()
	Start the tracking process.

cv::Point2f	imageToProjector (cv::Point2f imP, double z=0)
	Convert image coordinates to projector coordinates. More...

QPointF	imageToProjector (QPointF imP, double z=0)
	Convert image coordinates to projector coordinates (qt) More...

cv::Point2f	projectorToImage (cv::Point3d prP)
	projectorToImage converts projector (world) coords to image coords More...

void	updateProjectorCoordinates (std::vector< cv::Point2f > corners)
	Set the in-image coordinates of the projector corners.

void	setProjSize (const QSize &projSize)
	Set the size of the projector (in real units)

void	refreshCameraMatrix ()
	Update the camera matrices.

void	refreshProjectorMatrices ()
	Update the projector transformation matrices.

bool	cameraValid (int idx)
	Check if a camera index is valid. More...

Private Member Functions
void	timerEvent (QTimerEvent *)
	Process a timer event: Read the next frame, process it, emit it. More...

Private Attributes
QBasicTimer	mTimer
	Timer for processing frames.

QScopedPointer< cv::VideoCapture >	mVideoHandle
	The handler for the input video stream.

int	mFov
	The FOV of the camera. More...

QSize	mVideoSize
	The dimensions of the video stream.

QSize	mProjSize
	The dimensions (in real world units) of the projector screen.

double	mBallZ
	The distance of the ball plane from the projector plane (in real world units)

cv::Mat	mProjCornersCamera
	The location of projector coordinates in world (real units) More...

std::vector< cv::Point2f >	mCorners
	The location of projector coordinates in the image. More...

std::vector< cv::Point2f >	mBallPlane
	The bounding polygon of image coordinates which project to ball coordinates that live in the projector screen.

cv::Mat	mCameraMatrix
	The intrinsic camera matrix.

cv::Mat	mRMatrix
	The camera's extrinsic(?) rotation matrix.

cv::Mat	mTVector
	The camera's extrinsic(?) translation vector.

cv::Mat	mFrame
	The present frame received from the video stream.

DisplayFrameType	mEmitFrameType
	The type (e.g. More...

bool	mStarted = false
	Whether the video stream is started.

bool	mProjReady = false
	Whether there is enough projector data for transform.

bool	mStreamPaused = false
	Whether the stream is currently paused.

int	mFps
	The FPS of the video stream (=0 for webcam stream)

KFBallTracker	mTracker
	The ball tracker; we give it frames, it gives us ball predictions.

bool	mDisplayVideo = true
	Whether we should display video or not.

QPolygonF	mProjScreen
	Region to crop to if mClipTrack is on.

bool	mClipTrack = false
	Whether we should clip tracking to the proj screen.

Detailed Description

An object which owns some sort of input video stream, manages running it (and any tracking, etc) and emits cv::Mat frames and tracked objects.

Member Function Documentation

bool TrackingStream::cameraValid ( int idx )

Check if a camera index is valid.

Presently this does nothing... V4L detection on Linux seems a bit finicky and making this cross platform is an interesting problem. For now, it would seem reasonable if the program just be restarted in order to fix any issues here.

void TrackingStream::changeBallZ ( double z )

slot

changeBallZ sets the z-plane that the ball is thrown in (in inches)

Parameters

z	the z-distance from the projector plane in which the ball lies

void TrackingStream::changeBlurSize ( double blurSize )

slot

Set the strength of blur in preprocessing step.

Parameters

blurSize

void TrackingStream::changeGravConstant ( double g )

slot

Set the accel due to gravity g.

Parameters

g

void TrackingStream::changeMaxRadius ( double radius )

slot

Set the max viable radius of a ball blob.

Parameters

radius

void TrackingStream::changeMinRadius ( double radius )

slot

Set the min viable radius of a ball blob.

Parameters

radius

void TrackingStream::changeProjHeight ( int height )

slot

changeProjHeight sets the height of the projector screen (in inches)

Parameters

height the height of the projector screen

void TrackingStream::changeProjWidth ( int width )

slot

changeProjWidth sets the width of the projector screen (in inches)

Parameters

width the width of the projector screen

void TrackingStream::changeThreshVal ( double thresh )

slot

Set the threshold in preprocessing step.

Parameters

thresh

void TrackingStream::changeXYCovariance ( double sigma )

slot

Set the covariance in predicted X, Y (a KF parameter)

Parameters

sigma

cv::Point2f TrackingStream::imageToProjector	(	cv::Point2f	imP,
		double	z = `0`
	)

Convert image coordinates to projector coordinates.

Like most things, converting image coordinates (i.e. what the camera sees) to projector coordinates (world coordinates, where the top left corner of the projector is the origin) is just linear algebra.

This is the inverse of projectorToImage

The players:

R = mRMatrix, the camera rotation matrix
C = mCameraMatrix, the intrinsic camera matrix
T = mTVector, the camera translation vector
p = [imP, 1], the image point in homogeneous coordinates
z = the (a priori known) z-coordinate of the point in world coords
q = [X, Y, z] the world coordinates of the point p
s = Scaling factor that comes from transformation... picks appropriate point on the line projecting to p with the input z coordinate

$s = (z + (R^{-1}T)_{2,0}) / (R^{-1}C^{-1}p)_{2,0}$

$q = R^{-1}(sC^{-1}p - T)$

Parameters

imP	image point (x,y) to translate to world coordinates
z	a priori known world z-coordinate of the point to find

Returns

QPointF TrackingStream::imageToProjector	(	QPointF	imP,
		double	z = `0`
	)

Convert image coordinates to projector coordinates (qt)

A wrapper that works with Qt objects instead of opencv objects

Parameters

imP	image point (x,y) to translate to world coordinates
z	a priori known world z-coordinate of the point to find

Returns

cv::Point2f TrackingStream::projectorToImage ( cv::Point3d prP )

projectorToImage converts projector (world) coords to image coords

Parameters

prP	the point in world coordinates

Returns: the image pixel corresponding to prP

void TrackingStream::timerEvent ( QTimerEvent * ev )

private

Process a timer event: Read the next frame, process it, emit it.

Member Data Documentation

std::vector<cv::Point2f> TrackingStream::mCorners

private

The location of projector coordinates in the image.

Order is important: expected NW, SW, SE, NE (counterclockwise)

DisplayFrameType TrackingStream::mEmitFrameType

private

The type (e.g.

video, thresh...) of frame to emit

int TrackingStream::mFov

private

The FOV of the camera.

A PS2 Eye has two settings:

Red dot: 56deg, close up
Blue dot: 75deg, wide view

The smaller FOV is expected to cause less distortion, although it is likely to have less in the frame.

cv::Mat TrackingStream::mProjCornersCamera

private

The location of projector coordinates in world (real units)

Order is important: expected NW, SW, SE, NE (counterclockwise)

The documentation for this class was generated from the following files:

TrackingStream.hpp
TrackingStream.cpp

Public Slots

Signals

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

Member Function Documentation

Member Data Documentation