ffmpeg之AVPicture、AVFrame

2990阅读 0评论2017-09-20 leiminchn
分类:C/C++

1. AVPicture简介


  AVPicture是图片数据存储结构体。最多可以存储4个组件,最后一个是α
  AVPicturek可以在栈上创建。


2. AVPicture定义
 

点击(此处)折叠或打开

  1. typedef struct AVPicture {
  2.     uint8_t* data[AV_NUM_DATA_POINTERS]; ///< pointers to the image data planes
  3.     int linesize[AV_NUM_DATA_POINTERS]; ///< number of bytes per line
  4. } AVPicture;

3. AVPicture字段说明
    
    uint8_t
* data[AV_NUM_DATA_POINTERS] :指向解码后的图片数据的指针。对于视频来说是YUV/RGB数据,对于音频来说是PCM数据。
    int linesize[AV_NUM_DATA_POINTERS] :每行数据的大小。对于视频数据,行宽必须是16/32字节对齐(取决于cpu), 因此未必等于图片的宽,一般大于图像的宽。

4.AVPicture数据存储分布

   YUV420p:
     data[0]存储全部的Y数据,data[1]存储全部的U数据,data[2]存储全部的V数据。
      linesize[0]指明每行Y数据的宽度,linesize[1]指明每行U数据的宽度,linesize[2]指明每行U数据的宽度。
      每行数据大小计算公式 : line = width + padding size(补足16/32字节对齐,取决于cpu)。
      y数据的行数为分辨率高度hegith,u、v数据的行数为分辨率高度的二分之一即height/2。
  因此,所有数据大小buff= linesize[0] * height + linesize[1] * height/2 + linesize[2] * height/2。 

   RGB:
     data[0]存储全部RGB(A)数据,data[1],data[2],data[3]无效,因此linesize[1],linesize[2],linesize[3]也是无效的。
      每行数据大小计算公式: line = width *3 (无alpha通道,e.g rgb24); line = width * 4 (有alpha通道, e.g rgb32)。
 
      所有数据大小buff = linesize[0] * height。

   PCM:

      
  

  


1.AVFrame简介
  
  AVFrame存储解码后的原始音频或视频数据。
   创建必须使用av_frame_alloc, 该函数只创建AVFrame本身,数据缓冲区由其他方式管理;释放必须使用av_frame_free。AVFrame可以一次创建,重复使用。
   其内部的缓冲区通常由ffmpeg通过AVBuff API计数管理。
   sizeof(AVFrame) 的大小不是public ABI的一部分,因此不在栈上创建AVFrame。
   AVPicture是AVFrame的一个子集。
  

2.AVFrame定义
    

点击(此处)折叠或打开

  1. typedef struct AVFrame {
  2. #define AV_NUM_DATA_POINTERS 8
  3.     /**
  4.      * pointer to the picture/channel planes.
  5.      * This might be different from the first allocated byte
  6.      *
  7.      * Some decoders access areas outside 0,0 - width,height, please
  8.      * see avcodec_align_dimensions2(). Some filters and swscale can read
  9.      * up to 16 bytes beyond the planes, if these filters are to be used,
  10.      * then 16 extra bytes must be allocated.
  11.      */
  12.     uint8_t* data[AV_NUM_DATA_POINTERS];

  13.     /**
  14.      * For video, size in bytes of each picture line.
  15.      * For audio, size in bytes of each plane.
  16.      *
  17.      * For audio, only linesize[0] may be set. For planar audio, each channel
  18.      * plane must be the same size.
  19.      *
  20.      * For video the linesizes should be multiples of the CPUs alignment
  21.      * preference, this is 16 or 32 for modern desktop CPUs.
  22.      * Some code requires such alignment other code can be slower without
  23.      * correct alignment, for yet other it makes no difference.
  24.      *
  25.      * @note The linesize may be larger than the size of usable data -- there
  26.      * may be extra padding present for performance reasons.
  27.      */
  28.     int linesize[AV_NUM_DATA_POINTERS];

  29.     /**
  30.      * pointers to the data planes/channels.
  31.      *
  32.      * For video, this should simply point to data[].
  33.      *
  34.      * For planar audio, each channel has a separate data pointer, and
  35.      * linesize[0] contains the size of each channel buffer.
  36.      * For packed audio, there is just one data pointer, and linesize[0]
  37.      * contains the total size of the buffer for all channels.
  38.      *
  39.      * Note: Both data and extended_data should always be set in a valid frame,
  40.      * but for planar audio with more channels that can fit in data,
  41.      * extended_data must be used in order to access all channels.
  42.      */
  43.     uint8_t** extended_data;

  44.     /**
  45.      * width and height of the video frame
  46.      */
  47.     int width, height;

  48.     /**
  49.      * number of audio samples (per channel) described by this frame
  50.      */
  51.     int nb_samples;

  52.     /**
  53.      * format of the frame, -1 if unknown or unset
  54.      * Values correspond to enum AVPixelFormat for video frames,
  55.      * enum AVSampleFormat for audio)
  56.      */
  57.     int format;

  58.     /**
  59.      * 1 -> keyframe, 0-> not
  60.      */
  61.     int key_frame;

  62.     /**
  63.      * Picture type of the frame.
  64.      */
  65.     enum AVPictureType pict_type;

  66. #if FF_API_AVFRAME_LAVC
  67.     attribute_deprecated
  68.     uint8_t* base[AV_NUM_DATA_POINTERS];
  69. #endif

  70.     /**
  71.      * Sample aspect ratio for the video frame, 0/1 if unknown/unspecified.
  72.      */
  73.     AVRational sample_aspect_ratio;

  74.     /**
  75.      * Presentation timestamp in time_base units (time when frame should be shown to user).
  76.      */
  77.      int64_t pts;

  78.     /**
  79.      * PTS copied from the AVPacket that was decoded to produce this frame.
  80.      */
  81.      int64_t pkt_pts;

  82.     /**
  83.      * DTS copied from the AVPacket that triggered returning this frame. (if frame threading isn't used)
  84.      * This is also the Presentation time of this AVFrame calculated from
  85.      * only AVPacket.dts values without pts values.
  86.      */
  87.     int64_t pkt_dts;

  88.     /**
  89.      * picture number in bitstream order
  90.      */
  91.     int coded_picture_number;
  92.     /**
  93.      * picture number in display order
  94.      */
  95.     int display_picture_number;

  96.     /**
  97.      * quality (between 1 (good) and FF_LAMBDA_MAX (bad))
  98.      */
  99.     int quality;

  100. #if FF_API_AVFRAME_LAVC
  101.     attribute_deprecated
  102.     int reference;

  103.     /**
  104.      * QP table
  105.      */
  106.     attribute_deprecated
  107.     int8_t* qscale_table;
  108.     /**
  109.      * QP store stride
  110.      */
  111.     attribute_deprecated
  112.     int qstride;

  113.     attribute_deprecated
  114.     int qscale_type;

  115.     /**
  116.      * mbskip_table[mb]>=1 if MB didn't change
  117.      * stride= mb_width = (width+15)>>4
  118.      */
  119.     attribute_deprecated
  120.     uint8_t* mbskip_table;

  121.     /**
  122.      * motion vector table
  123.      * @code
  124.      * example:
  125.      * int mv_sample_log2= 4 - motion_subsample_log2;
  126.      * int mb_width= (width+15)>>4;
  127.      * int mv_stride= (mb_width << mv_sample_log2) + 1;
  128.      * motion_val[direction][x + y*mv_stride][0->mv_x, 1->mv_y];
  129.      * @endcode
  130.      */
  131.      int16_t(*motion_val[2])[2];

  132.     /**
  133.      * macroblock type table
  134.      * mb_type_base + mb_width + 2
  135.      */
  136.     attribute_deprecated
  137.     uint32_t* mb_type;

  138.     /**
  139.      * DCT coefficients
  140.      */
  141.     attribute_deprecated
  142.     short* dct_coeff;

  143.     /**
  144.      * motion reference frame index
  145.      * the order in which these are stored can depend on the codec.
  146.      */
  147.     attribute_deprecated
  148.     int8_t* ref_index[2];
  149. #endif

  150.     /**
  151.      * for some private data of the user
  152.      */
  153.     void* opaque;

  154.     /**
  155.      * error
  156.      */
  157.     uint64_t error[AV_NUM_DATA_POINTERS];

  158. #if FF_API_AVFRAME_LAVC
  159.     attribute_deprecated
  160.     int type;
  161. #endif

  162.     /**
  163.      * When decoding, this signals how much the picture must be delayed.
  164.      * extra_delay = repeat_pict / (2*fps)
  165.      */
  166.     int repeat_pict;

  167.     /**
  168.      * The content of the picture is interlaced.
  169.      */
  170.     int interlaced_frame;

  171.     /**
  172.      * If the content is interlaced, is top field displayed first.
  173.      */
  174.     int top_field_first;

  175.     /**
  176.      * Tell user application that palette has changed from previous frame.
  177.      */
  178.     int palette_has_changed;

  179. #if FF_API_AVFRAME_LAVC
  180.     attribute_deprecated
  181.     int buffer_hints;

  182.     /**
  183.      * Pan scan.
  184.      */
  185.     attribute_deprecated
  186.     struct AVPanScan* pan_scan;
  187. #endif

  188.     /**
  189.      * reordered opaque 64bit (generally an integer or a double precision float
  190.      * PTS but can be anything).
  191.      * The user sets AVCodecContext.reordered_opaque to represent the input at
  192.      * that time,
  193.      * the decoder reorders values as needed and sets AVFrame.reordered_opaque
  194.      * to exactly one of the values provided by the user through AVCodecContext.reordered_opaque
  195.      * @deprecated in favor of pkt_pts
  196.      */
  197.     int64_t reordered_opaque;

  198. #if FF_API_AVFRAME_LAVC
  199.     /**
  200.      * @deprecated this field is unused
  201.      */
  202.     attribute_deprecated
  203.     voidhwaccel_picture_private;

  204.     attribute_deprecated
  205.     struct AVCodecContext* owner;
  206.     attribute_deprecated
  207.     void* thread_opaque;

  208.     /**
  209.      * log2 of the size of the block which a single vector in motion_val represents:
  210.      * (4->16x16, 3->8x8, 2-> 4x4, 1-> 2x2)
  211.      */
  212.     uint8_t motion_subsample_log2;
  213. #endif

  214.     /**
  215.      * Sample rate of the audio data.
  216.      */
  217.     int sample_rate;

  218.     /**
  219.      * Channel layout of the audio data.
  220.      */
  221.     uint64_t channel_layout;

  222.     /**
  223.      * AVBuffer references backing the data for this frame. If all elements of
  224.      * this array are NULL, then this frame is not reference counted. This array
  225.      * must be filled contiguously -- if buf[i] is non-NULL then buf[j] must
  226.      * also be non-NULL for all j < i.
  227.      *
  228.      * There may be at most one AVBuffer per data plane, so for video this array
  229.      * always contains all the references. For planar audio with more than
  230.      * AV_NUM_DATA_POINTERS channels, there may be more buffers than can fit in
  231.      * this array. Then the extra AVBufferRef pointers are stored in the
  232.      * extended_buf array.
  233.      */
  234.     AVBufferRefbuf[AV_NUM_DATA_POINTERS];

  235.     /**
  236.      * For planar audio which requires more than AV_NUM_DATA_POINTERS
  237.      * AVBufferRef pointers, this array will hold all the references which
  238.      * cannot fit into AVFrame.buf.
  239.      *
  240.      * Note that this is different from AVFrame.extended_data, which always
  241.      * contains all the pointers. This array only contains the extra pointers,
  242.      * which cannot fit into AVFrame.buf.
  243.      *
  244.      * This array is always allocated using av_malloc() by whoever constructs
  245.      * the frame. It is freed in av_frame_unref().
  246.      */
  247.     AVBufferRef*extended_buf;
  248.     /**
  249.      * Number of elements in extended_buf.
  250.      */
  251.     int nb_extended_buf;

  252.     AVFrameSideData*side_data;
  253.     int nb_side_data;

  254. /**
  255.  * @defgroup lavu_frame_flags AV_FRAME_FLAGS
  256.  * Flags describing additional frame properties.
  257.  *
  258.  * @{
  259.  */

  260. /**
  261.  * The frame data may be corrupted, e.g. due to decoding errors.
  262.  */
  263. #define AV_FRAME_FLAG_CORRUPT (1 << 0)
  264. /**
  265.  * @}
  266.  */

  267.     /**
  268.      * Frame flags, a combination of @ref lavu_frame_flags
  269.      */
  270.     int flags;

  271.     /**
  272.      * MPEG vs JPEG YUV range.
  273.      * It must be accessed using av_frame_get_color_range() and
  274.      * av_frame_set_color_range().
  275.      * - encoding: Set by user
  276.      * - decoding: Set by libavcodec
  277.      */
  278.     enum AVColorRange color_range;

  279.     enum AVColorPrimaries color_primaries;

  280.     enum AVColorTransferCharacteristic color_trc;

  281.     /**
  282.      * YUV colorspace type.
  283.      * It must be accessed using av_frame_get_colorspace() and
  284.      * av_frame_set_colorspace().
  285.      * - encoding: Set by user
  286.      * - decoding: Set by libavcodec
  287.      */
  288.     enum AVColorSpace colorspace;

  289.     enum AVChromaLocation chroma_location;

  290.     /**
  291.      * frame timestamp estimated using various heuristics, in stream time base
  292.      * Code outside libavcodec should access this field using:
  293.      * av_frame_get_best_effort_timestamp(frame)
  294.      * - encoding: unused
  295.      * - decoding: set by libavcodec, read by user.
  296.      */
  297.      int64_t best_effort_timestamp;

  298.     /**
  299.      * reordered pos from the last AVPacket that has been input into the decoder
  300.      * Code outside libavcodec should access this field using:
  301.      * av_frame_get_pkt_pos(frame)
  302.      * - encoding: unused
  303.      * - decoding: Read by user.
  304.      */
  305.     int64_t pkt_pos;

  306.     /**
  307.      * duration of the corresponding packet, expressed in
  308.      * AVStream->time_base units, 0 if unknown.
  309.      * Code outside libavcodec should access this field using:
  310.      * av_frame_get_pkt_duration(frame)
  311.      * - encoding: unused
  312.      * - decoding: Read by user.
  313.      */
  314.     int64_t pkt_duration;

  315.     /**
  316.      * metadata.
  317.      * Code outside libavcodec should access this field using:
  318.      * av_frame_get_metadata(frame)
  319.      * - encoding: Set by user.
  320.      * - decoding: Set by libavcodec.
  321.      */
  322.     AVDictionary *metadata;

  323.     /**
  324.      * decode error flags of the frame, set to a combination of
  325.      * FF_DECODE_ERROR_xxx flags if the decoder produced a frame, but there
  326.      * were errors during the decoding.
  327.      * Code outside libavcodec should access this field using:
  328.      * av_frame_get_decode_error_flags(frame)
  329.      * - encoding: unused
  330.      * - decoding: set by libavcodec, read by user.
  331.      */
  332.     int decode_error_flags;
  333. #define FF_DECODE_ERROR_INVALID_BITSTREAM 1
  334. #define FF_DECODE_ERROR_MISSING_REFERENCE 2

  335.     /**
  336.      * number of audio channels, only used for audio.
  337.      * Code outside libavcodec should access this field using:
  338.      * av_frame_get_channels(frame)
  339.      * - encoding: unused
  340.      * - decoding: Read by user.
  341.      */
  342.     int channels;

  343.     /**
  344.      * size of the corresponding packet containing the compressed
  345.      * frame. It must be accessed using av_frame_get_pkt_size() and
  346.      * av_frame_set_pkt_size().
  347.      * It is set to a negative value if unknown.
  348.      * - encoding: unused
  349.      * - decoding: set by libavcodec, read by user.
  350.      */
  351.     int pkt_size;

  352.     /**
  353.      * Not to be accessed directly from outside libavutil
  354.      */
  355.     AVBufferRef* qp_table_buf;
  356. } AVFrame;


3.AVFrame字段说明

  uint8_t* data[AV_NUM_DATA_POINTERS] : 指向解码后的数据的指针数组。参考AVPicture字段说明。解码时,由ffmpeg相关解码函数设置;编码时由调用者设置(可以通过av_img_copy将解码后的数据复制到即将进行编码的AVFrame中)。
  int linesize[AV_NUM_DATA_POINTERS] : 指明每行数据的大小。参考AVPicture字段说明。解码时,由ffmpeg相关解码函数设置;编码时由调用者设置(可以通过av_img_copy将解码后的数据复制到即将进行编码的AVFrame中)。
  uint8_t*extended_data : 扩展数据缓冲区。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段

  int width : 视频帧的宽度。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  
int 
height : 视频帧的高度。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int nb_samples : 音频帧的数量解码时,由ffmpeg相关解码函数设置;编码时由调用者设置。
  int format :图片格式。-1时未设置或未知。对于视频数据而言是枚举类型AVPixelFormat(yuv402p,rgb24等),对于音频而言是枚举类型AVSampleFormat。 解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int key_frame : 是否为关键帧标志。为1时为关键帧,为0时为非关键帧。解码和编码时都由ffmpeg设置。
  enum AVPictureType pict_type :图片类型0。为枚举类型AVPictureType(I,B,P等)。解码和编码时都由ffmpeg设置。
  uint8_t* base[AV_NUM_DATA_POINTERS] :
  AVRational sample_aspect_ratio : 视频的宽高比。0时未知,1时未指定。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int64_t pts : 显示时间戳。解码时,由ffmpeg相关解码函数设置;编码时由调用者设置。
  int64_t pkt_pts : 拷贝自AVPakcet的显示时间戳。显示时,依赖该字段,而非 pts 字段。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段

  int64_t pkt_dts : 拷贝自AVPacket的解码时间戳。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int coded_picture_number : 图片在比特流序列中的序号。解码和编码时都由ffmpeg设置。
  int display_picture_number : 图片在显示序列中的序号。解码和编码时都由ffmpeg设置。
  int quality : 质量。处于1(good)至 FF_LAMBDA_MAX (bad))之间。解码和编码时都由ffmpeg设置。
  int reference :指明图片是否是引用的。解码和编码时都由ffmpeg设置。
  int8_t* qscale_table : QP表。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int qstride : QP存储步幅。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int qscale_type :
  uint8_t* mbskip_table : 跳过宏块表。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int16_t (*motion_val[2])[2] :运动矢量表。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  uint32_t* mb_type : 宏块类型表。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  short* dct_coeff : DCT系数。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  int8_t* ref_index[2] :参考帧列表。解码时由ffmpeg设置,供调用者读取, 编码时不使用该字段
  void* opaque : 用户私有数据。解码时由用户设置, 编码时不使用该字段
  uint64_t error[AV_NUM_DATA_POINTERS] : error。解码时不适用该字段,编码时由ffmpeg设置。
  int type : buffer的类型。解码和编码时都由分配该结构体者设置。
  int repeat_pict : 图片延迟显示。extra_delay=repeat_pict/(2*fps)。解码时由ffmpeg设置,供调用者读取,编码时不使用该字段。
  int interlaced_frame : 隔行扫描图片标志。解码时由ffmpeg设置,供调用者读取,默认为0,编码时由调用者指定。
  int top_field_first :如果隔行扫描图片,顶部字段首先显示标志。解码时由ffmpeg设置,供调用者读取,编码时由调用者指定。
  int palette_has_changed : 告知调用者,调色板是否从前一帧改变。解码时由ffmpeg设置,供调用者读取,默认为0,编码时不使用该字段。
  int buffer_hints : 建议buffer类型,不为0时有效。解码时由ffmpeg设置,供调用者读取,默认为0,编码时不使用该字段。
  struct AVPanScan* pan_scan :平移扫描标志。解码时由ffmpeg设置,供调用者读取,编码时由调用者指定。
  int64_t reordered_opaque :64位重新排列不透明值。 解码时由ffmpeg设置,供调用者读取,编码时不使用该字段。
  void* hwaccel_picture_private :硬件加速器私有数据。 解码时由ffmpeg设置,供调用者读取,编码时不使用该字段。
  struct AVCodecContext* owner : 指向最后一次调用ff_thread_get_buffer()的AVCodeContext指针。解码和编码时都由ffmpeg设置。
  void* thread_opaque : 多线程时存储特定帧信息。解码和编码时都由ffmpeg设置。
  uint8_t motion_subsample_log2 : 在motion_val中单个向量表示的块的大小的log2。 解码时由ffmpeg设置,供调用者读取,编码时不使用该字段。
  int sample_rate :音频采样率。 解码时由调用者设置,编码时不使用该字段。
  uint64_t channel_layout : 音频通道布局。解码时由调用者设置,编码时不使用该字段。
  AVBufferRef* buf[AV_NUM_DATA_POINTERS] :
  AVBufferRef** extended_buf :
  int nb_extended_buf : 
  AVFrameSideData** side_data :
  int  nb_side_data :
  int flags :
  enum AVColorRange color_range :
  enum AVColorPrimaries color_primaries :
  enum AVColorTransferCharacteristic color_trc :
  enum AVColorSpace colorspace :
  enum AVChromaLocation chroma_location :
  int64_t best_effort_timestamp : 在时间基础上使用各种启发式估计的帧时间戳。libavcodec之外的代码应使用av_frame_get_best_effort_timestamp(frame)访问此字段。解码时由ffmpeg设置,供调用者读取,编码时不使用该字段。
  int64_t pkt_pos :
  int64_t pkt_duration :
  AVDictionary* metadata:
  int decode_error_flags:
  int channels :
  int pkt_size :
  AVBufferRef* qp_table_buf :
  
上一篇:ffmpeg之AVPacket
下一篇:h264中I帧的判断