Phytec中国的wiki
support@phytec.cn
热线:0755-61802110-803
Gstreamer混合图像处理会使用videobox, alpha, videomixer等插件, 以下是对这些插件的介绍和测试demo.
videobox
这个插件可以用来增大或者裁减图形. videobox有四个属性做为输入,即top,bottom, left和right偏移值. 正数值表示从图像的边框裁减相应的像素值. 而负数值代表增加相应的像素值, 也可以理解为增加了边框, 增加边框的颜色可通过fill参数来指定, 或者同过alpha插件来设置内部图片和边框的透明度.
更多videobox可参考官方文档:
以下是videobox的两个pipeline, 可供参考
root@phyboard-mira-imx6-3:~# gst-launch-1.0 -v videotestsrc pattern=snow ! video/x-raw, width=600, height=400 ! videobox top=-50 left=-50 fill=blue ! fbdevsink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mo no, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoBox:videobox0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)650, height=(int)450, framerate=(fraction)30/1, multiview-mode=(string)mono, pixe l-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstFBDEVSink:fbdevsink0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)650, height=(int)450, framerate=(fraction)30/1, multiview-mode=(string)mono, p ixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoBox:videobox0.GstPad:sink: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pix el-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:02.187046334 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... root@phyboard-mira-imx6-3:~# gst-launch-1.0 -v videotestsrc ! video/x-raw, width=600, height=400 ! videobox top=50 left=50 ! fbdevsink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mo no, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoBox:videobox0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)550, height=(int)350, framerate=(fraction)30/1, multiview-mode=(string)mono, pixe l-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstFBDEVSink:fbdevsink0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)550, height=(int)350, framerate=(fraction)30/1, multiview-mode=(string)mono, p ixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoBox:videobox0.GstPad:sink: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pix el-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)AYUV, width=(int)600, height=(int)400, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock
alpha
alpha插件可以用来设置透明度, alpha值为0.0时为总透明, 1.0为不透明.
gstreamer alpha插件文档: https://gstreamer.freedesktop.org/documentation/alpha/index.html?gi-language=c
gst-launch-1.0 \ videomixer name=mix ! videoconvert ! autovideosink \ -v videotestsrc pattern="snow" ! "video/x-raw,format=UYVY,width=640,height=480" ! alpha alpha=1.0 ! mix. \ -v videotestsrc ! "video/x-raw,format=UYVY,width=320,height=240" ! alpha alpha=1.0 ! videobox top=-80 left=-80 border-alpha=0 ! mix.
videomixer
gstreamer videomixer插件文档: https://gstreamer.freedesktop.org/documentation/videomixer/index.html?gi-language=c
videomixer可以混合不同格式的视屏流.这个插件会比较每个输入视屏流的分辨率和帧率来定义输出视屏流的参数,输出视屏流的分辨率是输入视频流中的最大值, 而输出帧率是输入视屏流中的最快帧率.
这是在Miraq开发板上测试过的显示4个模拟摄像头视频流的pipeline, 可供参考:
gst-launch-1.0 \ videomixer name=mix sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=960 sink_1::ypos=0 sink_2::xpos=0 sink_2::ypos=540 sink_3::xpos=960 sink_3::ypos=540 ! videoconvert ! kmssink driver-name=imx-drm \ -v v4l2src device=/dev/video8 ! video/x-raw,format=UYVY,depth=8,interlace-mode=interleaved ! deinterlace ! videoscale ! video/x-raw,depth=8,width=960,height=540 ! mix.sink_0 \ -v v4l2src device=/dev/video9 ! video/x-raw,format=UYVY,depth=8,interlace-mode=interleaved ! deinterlace ! videoscale ! video/x-raw,depth=8,width=960,height=540 ! mix.sink_1 \ -v v4l2src device=/dev/video10 ! video/x-raw,format=UYVY,depth=8,interlace-mode=interleaved ! deinterlace ! videoscale ! video/x-raw,depth=8,width=960,height=540 ! mix.sink_2 \ -v v4l2src device=/dev/video11 ! video/x-raw,format=UYVY,depth=8,interlace-mode=interleaved ! deinterlace ! videoscale ! video/x-raw,depth=8,width=960,height=540 ! mix.sink_3
我们测试过程中发现videomixer输出的视屏流的扫描模式(interlace-mode)为逐行扫描方式(progressive), 如果直接混合多个隔行扫描方式(interleaved)的视频流, gstreamer会报错, 所以在混合多个隔行扫描的视频流时,需要先使用deinterlace插件将视频流的扫描方式转换为逐行扫描方式, 然后再进行混合视频流.