Android/scrcpy
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity
scrcpy/app/src/decoder.c

161 lines
4.3 KiB
C
Raw Normal View History

Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include "decoder.h"
#include <libavformat/avformat.h>
#include "events.h"
Rename "frames" to "video_buffer" It better describes the purpose of the structure.
2019-03-02 22:16:55 +08:00
#include "video_buffer.h"
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
#include "trait/frame_sink.h"
Move utilities to util/
2019-11-24 18:53:00 +08:00
#include "util/log.h"
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
/** Downcast packet_sink to decoder */
#define DOWNCAST(SINK) container_of(SINK, struct decoder, packet_sink)
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static void
decoder_close_first_sinks(struct decoder *decoder, unsigned count) {
while (count) {
struct sc_frame_sink *sink = decoder->sinks[--count];
sink->ops->close(sink);
}
}
static inline void
decoder_close_sinks(struct decoder *decoder) {
decoder_close_first_sinks(decoder, decoder->sink_count);
}
static bool
decoder_open_sinks(struct decoder *decoder) {
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->open(sink)) {
LOGE("Could not open frame sink %d", i);
decoder_close_first_sinks(decoder, i);
return false;
}
}
return true;
}
static bool
Add more consts Some decoder and recorder functions must not write to AVCodec and AVPacket.
2019-03-03 18:59:31 +08:00
decoder_open(struct decoder *decoder, const AVCodec *codec) {
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
decoder->codec_ctx = avcodec_alloc_context3(codec);
if (!decoder->codec_ctx) {
Define macros wrappers for logs Use macros to wrap SDL_Log* functions with the "application" category.
2018-02-13 17:10:18 +08:00
LOGC("Could not allocate decoder context");
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
if (avcodec_open2(decoder->codec_ctx, codec, NULL) < 0) {
LOGE("Could not open codec");
avcodec_free_context(&decoder->codec_ctx);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
decoder->frame = av_frame_alloc();
if (!decoder->frame) {
LOGE("Could not create decoder frame");
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
return false;
}
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
if (!decoder_open_sinks(decoder)) {
LOGE("Could not open decoder sinks");
av_frame_free(&decoder->frame);
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
return false;
}
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
}
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static void
Update code style Limit source code to 80 chars, and declare functions return type and modifiers on a separate line. This allows to avoid very long lines, and all function names are aligned. (We do this on VLC, and I like it.)
2019-03-03 03:09:56 +08:00
decoder_close(struct decoder *decoder) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
decoder_close_sinks(decoder);
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
av_frame_free(&decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
}
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
push_frame_to_sinks(struct decoder *decoder, const AVFrame *frame) {
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->push(sink, frame)) {
LOGE("Could not send frame to sink %d", i);
return false;
}
}
return true;
}
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
Add more consts Some decoder and recorder functions must not write to AVCodec and AVPacket.
2019-03-03 18:59:31 +08:00
decoder_push(struct decoder *decoder, const AVPacket *packet) {
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
bool is_config = packet->pts == AV_NOPTS_VALUE;
if (is_config) {
// nothing to do
return true;
}
Handle EAGAIN on send_packet in decoder EAGAIN was only handled on receive_frame. In practice, it should not be necessary, since one packet always contains one frame. But just in case.
2021-04-18 23:13:58 +08:00
int ret = avcodec_send_packet(decoder->codec_ctx, packet);
if (ret < 0 && ret != AVERROR(EAGAIN)) {
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
LOGE("Could not send video packet: %d", ret);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
ret = avcodec_receive_frame(decoder->codec_ctx, decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
if (!ret) {
// a frame was received
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
bool ok = push_frame_to_sinks(decoder, decoder->frame);
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
// A frame lost should not make the whole pipeline fail. The error, if
// any, is already logged.
(void) ok;
Unref decoder AVFrame immediately The frame can be unref immediately after it is pushed to the frame sinks. It was not really a memory leak because the frame was unref every time by avcodec_receive_frame() (and freed on close), but a reference was unnecessarily kept for too long.
2021-04-27 00:05:43 +08:00
av_frame_unref(decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
} else if (ret != AVERROR(EAGAIN)) {
LOGE("Could not receive video frame: %d", ret);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
}
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static bool
decoder_packet_sink_open(struct sc_packet_sink *sink, const AVCodec *codec) {
struct decoder *decoder = DOWNCAST(sink);
return decoder_open(decoder, codec);
}
static void
decoder_packet_sink_close(struct sc_packet_sink *sink) {
struct decoder *decoder = DOWNCAST(sink);
decoder_close(decoder);
}
static bool
decoder_packet_sink_push(struct sc_packet_sink *sink, const AVPacket *packet) {
struct decoder *decoder = DOWNCAST(sink);
return decoder_push(decoder, packet);
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
void
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
decoder_init(struct decoder *decoder) {
Explicitly initialize decoder sink_count The zero-initialization relied on the fact that the decoder instance is static.
2021-05-29 03:23:10 +08:00
decoder->sink_count = 0;
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static const struct sc_packet_sink_ops ops = {
.open = decoder_packet_sink_open,
.close = decoder_packet_sink_close,
.push = decoder_packet_sink_push,
};
decoder->packet_sink.ops = &ops;
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
}
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
void
decoder_add_sink(struct decoder *decoder, struct sc_frame_sink *sink) {
assert(decoder->sink_count < DECODER_MAX_SINKS);
assert(sink);
assert(sink->ops);
decoder->sinks[decoder->sink_count++] = sink;
}
Reference in a new issue Copy permalink