Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

下面这个表格介绍了一下常用的功能,gstreamer可实现的功能远不止这些。

功能说明
USB的UVC摄像头/SOC摄像头接口gstreamer都提供支持
硬视频编解码我们平台通过v4l2设备来实现
软视频编解码大部分通过libav来实现
打包/解包如matroskamux,avimux,mp4mux
音频编解码

基本都支持,mp3,ogg等

rtp/rtsp可支持服务端,客户端,打包解包
测试信号源视频源 videotestsrc 可输出如雪花点,SMPTE测试图形等
读取/写入文件可方便的从文件读取作为源,或者将任意位置数据写入文件
显示可将视频输出到framebuffer或者drm设备
在软件中处理流的数据appsrc和appsink可以方便的在软件中生成和处理数据
将视频流分开两个流处理gtreamer可以很容易实现
音频视频分别处理天然支持

gstreamer命令行测试/调试工具

gstreamer有一系列工具可以很方便的用来验证想法,甚至直接作为应用。

常用的指令有两个:

命令介绍
gst-launch执行gstreamer pipeline,实现各种功能
gst-inspect查看所有的插件/处理模块,或查看模块的信息

如查找解码器:

Code Block
languagebash
themeMidnight
root@phyboard-mira-imx6-3:~# gst-inspect-1.0 | grep Decoder
video4linux2:  v4l2mpeg4dec: V4L2 MPEG4 Decoder
video4linux2:  v4l2mpeg2dec: V4L2 MPEG2 Decoder
video4linux2:  v4l2h264dec: V4L2 H264 Decoder
playback:  uridecodebin3: URI Decoder
playback:  uridecodebin: URI Decoder
playback:  decodebin3: Decoder Bin 3
playback:  decodebin: Decoder Bin
rtp:  rtpulpfecdec: RTP FEC Decoder
rtp:  rtpreddec: Redundant Audio Data (RED) Decoder
rtsp:  rtpdec: RTP Decoder

...

Code Block
languagebash
themeMidnight
titlegst-inspect-1.0 v4l2mpeg4dec
collapsetrue
root@phyboard-mira-imx6-3:~# gst-inspect-1.0 v4l2mpeg4dec
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                V4L2 MPEG4 Decoder
  Klass                    Codec/Decoder/Video
  Description              Decodes MPEG4 streams via V4L2 API
  Author                   Nicolas Dufresne <nicolas.dufresne@collabora.com>

Plugin Details:
  Name                     video4linux2
  Description              elements for Video 4 Linux
  Filename                 /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
  Version                  1.14.1
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2018-05-17
  Binary package           GStreamer Good Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstV4l2VideoDec
                               +----v4l2mpeg4dec

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/mpeg
            mpegversion: 4
           systemstream: false

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)I420, (string)YV12, (string)YUY2 }
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "v4l2mpeg4dec0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  device              : Device location
                        flags: readable
                        String. Default: "/dev/video9"
  device-name         : Name of the device
                        flags: readable
                        String. Default: "CODA960"
  device-fd           : File descriptor of the device
                        flags: readable
                        Integer. Range: -1 - 2147483647 Default: -1
  output-io-mode      : Output side I/O mode (matches sink pad)
                        flags: readable, writable
                        Enum "GstV4l2IOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (1): rw               - GST_V4L2_IO_RW
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (3): userptr          - GST_V4L2_IO_USERPTR
                           (4): dmabuf           - GST_V4L2_IO_DMABUF
                           (5): dmabuf-import    - GST_V4L2_IO_DMABUF_IMPORT
  capture-io-mode     : Capture I/O mode (matches src pad)
                        flags: readable, writable
                        Enum "GstV4l2IOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (1): rw               - GST_V4L2_IO_RW
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (3): userptr          - GST_V4L2_IO_USERPTR
                           (4): dmabuf           - GST_V4L2_IO_DMABUF
                           (5): dmabuf-import    - GST_V4L2_IO_DMABUF_IMPORT
  extra-controls      : Extra v4l2 controls (CIDs) for the device
                        flags: readable, writable
                        Boxed pointer of type "GstStructure"

...

Code Block
languagebash
themeMidnight
titlegst.ccollapsetrue
#include <gst/gst.h>
#include <stdio.h>
#include <signal.h>
#include <string.h>

#define CLIENT_IP "192.168.3.10"
#define TARGET_BITRATE 15000000
#define WIDTH 1280
#define HEIGHT 720

static GMainLoop *loop;
GstElement *pipeline, *source, *convert, *encoder, *rtp264, *sink, *capsfilter;
GstCaps *source_caps, *bayer_filter_caps, *YUV_filter_caps, *Ethernet_filter_caps;
GstBus *bus;

GstStateChangeReturn ret;
guint bus_watch_id;

static void sigint_restore (void)
{
        struct sigaction action;

        memset (&action, 0, sizeof (action));
        action.sa_handler = SIG_DFL;

        sigaction (SIGINT, &action, NULL);
}

/* Signal handler for ctrl+c */
void intHandler(int dummy) {
        //! Emit the EOS signal which tells all the elements to shut down properly:
        printf("Sending EOS signal to shutdown pipeline cleanly\n");
        gst_element_send_event(pipeline, gst_event_new_eos());
        sigint_restore();
        return;
}

static gboolean bus_call        (GstBus     *bus, GstMessage *msg, gpointer data)
{
        GMainLoop *loop = (GMainLoop *) data;

        switch (GST_MESSAGE_TYPE (msg)) {

                case GST_MESSAGE_EOS:
                        g_print ("End of stream\n");
                        g_main_loop_quit (loop);
                        break;

                case GST_MESSAGE_ERROR: {
                                                gchar  *debug;
                                                GError *error;

                                                gst_message_parse_error (msg, &error, &debug);
                                                g_free (debug);

                                                g_printerr ("Error: %s\n", error->message);
                                                g_error_free (error);

                                                g_main_loop_quit (loop);
                                                break;
                                        }

                default:
                                        break;
        }

        return TRUE;
}
int watcher_make()
{
        /* we add a message handler */
        bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
        bus_watch_id = gst_bus_add_watch (bus, bus_call, loop);
        gst_object_unref (bus);
        return 0;
}
                      
int main(int argc, char *argv[]) {
        signal(SIGINT, intHandler);

        /* Initialize GStreamer */
        gst_init (&argc, &argv);
        loop = g_main_loop_new (NULL, FALSE);

        /* Create the elements */
        //      source = gst_element_factory_make ("appsrc", "source");
        source   = gst_element_factory_make ("v4l2src","source");
        convert  = gst_element_factory_make ("videoconvert","convert");
        encoder  = gst_element_factory_make ("v4l2h264enc","encoder");
        rtp264   = gst_element_factory_make ("rtph264pay","rtp264");
        sink     = gst_element_factory_make ("udpsink","sink");
        capsfilter = gst_element_factory_make ("capsfilter","video_caps");

        /* Create the empty pipeline */
        pipeline = gst_pipeline_new ("pipeline");

        if (!pipeline || !source || !encoder || !rtp264 || !sink) {
                g_printerr ("Not all elements could be created.\n");
                return -1;
        }

        /* Modify the source properties */
        g_object_set (G_OBJECT (source), "device", "/dev/video10", NULL);

        /* Modify the caps properties */
        source_caps = gst_caps_from_string ("video/x-raw,width=1280,height=720,colorimetry=bt709");
        g_object_set (G_OBJECT (capsfilter), "caps", source_caps, NULL);
        gst_caps_unref(source_caps);

        g_object_set( G_OBJECT(rtp264), "pt", 96, "config-interval",10,NULL);
        g_object_set( G_OBJECT(sink), "host", CLIENT_IP, "port", 5000, "sync",FALSE, NULL);

        /* Add function to watch bus */
        if(watcher_make() != 0)
                return -1;

        /* Build the pipeline */
        gst_bin_add_many (GST_BIN (pipeline), source, convert, capsfilter, encoder, rtp264, sink, NULL);

        /* Link the elements together */
        gst_element_link_many (source, convert, capsfilter, encoder, rtp264, sink, NULL);


        /* Start playing */
        ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
        if (ret == GST_STATE_CHANGE_FAILURE) {
                g_printerr ("Unable to set the pipeline to the playing state.\n");
                gst_object_unref (pipeline);
                return -1;
        }
        g_main_loop_run(loop);


        /* Free resources */

        gst_element_set_state (pipeline, GST_STATE_NULL);
        gst_object_unref (GST_OBJECT (pipeline));
        g_main_loop_unref (loop);

        return 0;
}

...

Code Block
languagebash
themeMidnight
root@phyboard-mira-imx6-3:~# ls gstreamer_examples 该目录下主要是一些操作相机的gstreamer指令
bwcam-fbdev_640x480.sh          func.sh                         phytec_usb_cam                  register-settings-tw9910.txt
bwcam-save_jpg_full_res.sh      master-clock_vm006.sh           port_cam-1_via_ipu2-csi1        register-settings-vita1300.txt
bwcam-save_raw_full_res.sh      more_mt9m131_scripts            readme.txt                      remove_qt_demo.sh
camera_select_i.MX6.txt         more_mt9p031_scripts            register-settings-mt9m001.txt   tools
colcam-fbdev_640x480.sh         more_mt9v024_scripts            register-settings-mt9m131.txt   vpu_enc_dec_scripts
colcam-save_jpg_full_res.sh     more_vita1300_scripts           register-settings-mt9p031.txt
colcam-save_raw_full_res.sh     phytec_thermal_cam              register-settings-mt9v02x.txt

root@phyboard-mira-imx6-3:~# ls gstreamer_examples/vpu_enc_dec_scripts/ 操作VPU的gstreamer指令
VLC_Setup.txt             bwcam_network_stream.sh   colcam_network_stream.sh  readme.txt
bwcam-save_h264.sh        colcam-save_h264.sh       play_h264_mkv-file.sh     show_network_stream.sh

root@phyboard-mira-imx6-3:~# ls gstreamer_examples/phytec_usb_cam/ USB摄像头的gstreamer指令
func.sh                         usb_bwcam-fbdev_640x480.sh      usb_bwcam-save_raw_full_res.sh
readme.txt                      usb_bwcam-save_jpg_full_res.sh

root@phyboard-mira-imx6-3:~# ls v4l2_c-examples 使用v4l2来操作相机的指令
anleitung_instruction.txt      mt9p031_bw_full_save-raw.sh    vita1300_bw_full_save-raw.sh
mt9m131_col_full_save-raw.sh   mt9v02x_bw_full_save-raw.sh    vita1300_col_full_save-raw.sh
mt9p006_col_full_save-raw.sh   mt9v02x_col_full_save-raw.sh   vm05x_full_save-raw.sh

也可以在这里下载:


我们的i.MX6软件平台与NXP方案的区别

我们使用开源的VPU驱动以及开源的gstreamer插件,如图:

...