Android/scrcpy
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity
scrcpy/app/src/decoder.c

110 lines
2.8 KiB
C
Raw Normal View History

Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include "decoder.h"
Add missing includes Refs #2616 <https://github.com/Genymobile/scrcpy/issues/2616>
2021-09-21 00:27:37 +08:00
#include <libavcodec/avcodec.h>
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include <libavformat/avformat.h>
Add audio player Play the decoded audio using SDL. The audio player frame sink receives the audio frames, resample them and write them to a byte buffer (introduced by this commit). On SDL audio callback (from an internal SDL thread), copy samples from this byte buffer to the SDL audio buffer. The byte buffer is protected by the SDL_AudioDeviceLock(), but it has been designed so that the producer and the consumer may write and read in parallel, provided that they don't access the same slices of the ring-buffer buffer. PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757> Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-03 07:43:20 +08:00
#include <libavutil/channel_layout.h>
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include "events.h"
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
#include "trait/frame_sink.h"
Move utilities to util/
2019-11-24 18:53:00 +08:00
#include "util/log.h"
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
/** Downcast packet_sink to decoder */
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
#define DOWNCAST(SINK) container_of(SINK, struct sc_decoder, packet_sink)
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
sc_decoder_open(struct sc_decoder *decoder, AVCodecContext *ctx) {
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
decoder->frame = av_frame_alloc();
if (!decoder->frame) {
Add OutOfMemory log helper Add a special LOG_OOM() function to log all OutOfMemory errors (i.e. allocations returning NULL).
2021-11-25 05:06:11 +08:00
LOG_OOM();
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
return false;
}
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
if (!sc_frame_source_sinks_open(&decoder->frame_source, ctx)) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
av_frame_free(&decoder->frame);
return false;
}
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
decoder->ctx = ctx;
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
}
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_close(struct sc_decoder *decoder) {
Use frame source trait in decoder
2023-03-02 16:37:36 +08:00
sc_frame_source_sinks_close(&decoder->frame_source);
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
av_frame_free(&decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
}
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
bool is_config = packet->pts == AV_NOPTS_VALUE;
if (is_config) {
// nothing to do
return true;
}
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
int ret = avcodec_send_packet(decoder->ctx, packet);
Handle EAGAIN on send_packet in decoder EAGAIN was only handled on receive_frame. In practice, it should not be necessary, since one packet always contains one frame. But just in case.
2021-04-18 23:13:58 +08:00
if (ret < 0 && ret != AVERROR(EAGAIN)) {
Give a name to decoder instances This will be useful in logs. PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-02-25 04:22:35 +08:00
LOGE("Decoder '%s': could not send video packet: %d",
decoder->name, ret);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Call avcodec_receive_frame() in a loop Since in scrcpy a video packet passed to avcodec_send_packet() is always a complete video frame, it is sufficient to call avcodec_receive_frame() exactly once. In practice, it also works for audio packets: the decoder produces exactly 1 frame for 1 input packet. In theory, it is an implementation detail though, so avcodec_receive_frame() should be called in a loop.
2023-03-01 04:20:28 +08:00
for (;;) {
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
ret = avcodec_receive_frame(decoder->ctx, decoder->frame);
Call avcodec_receive_frame() in a loop Since in scrcpy a video packet passed to avcodec_send_packet() is always a complete video frame, it is sufficient to call avcodec_receive_frame() exactly once. In practice, it also works for audio packets: the decoder produces exactly 1 frame for 1 input packet. In theory, it is an implementation detail though, so avcodec_receive_frame() should be called in a loop.
2023-03-01 04:20:28 +08:00
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
break;
}
if (ret) {
LOGE("Decoder '%s', could not receive video frame: %d",
decoder->name, ret);
return false;
}
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
// a frame was received
Use frame source trait in decoder
2023-03-02 16:37:36 +08:00
bool ok = sc_frame_source_sinks_push(&decoder->frame_source,
decoder->frame);
Unref decoder AVFrame immediately The frame can be unref immediately after it is pushed to the frame sinks. It was not really a memory leak because the frame was unref every time by avcodec_receive_frame() (and freed on close), but a reference was unnecessarily kept for too long.
2021-04-27 00:05:43 +08:00
av_frame_unref(decoder->frame);
Stop on decoder frame push error On push, frame sinks report downstream errors to stop upstream components. Do not ignore the error.
2023-03-03 07:42:51 +08:00
if (!ok) {
// Error already logged
return false;
}
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
}
Call avcodec_receive_frame() in a loop Since in scrcpy a video packet passed to avcodec_send_packet() is always a complete video frame, it is sufficient to call avcodec_receive_frame() exactly once. In practice, it also works for audio packets: the decoder produces exactly 1 frame for 1 input packet. In theory, it is an implementation detail though, so avcodec_receive_frame() should be called in a loop.
2023-03-01 04:20:28 +08:00
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static bool
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
sc_decoder_packet_sink_open(struct sc_packet_sink *sink, AVCodecContext *ctx) {
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
struct sc_decoder *decoder = DOWNCAST(sink);
Pass AVCodecContext to packet sinks Create the codec context from the demuxer, so that it can fill context data for the decoder and recorder.
2023-03-11 02:25:45 +08:00
return sc_decoder_open(decoder, ctx);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
static void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_packet_sink_close(struct sc_packet_sink *sink) {
struct sc_decoder *decoder = DOWNCAST(sink);
sc_decoder_close(decoder);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_packet_sink_push(struct sc_packet_sink *sink,
const AVPacket *packet) {
struct sc_decoder *decoder = DOWNCAST(sink);
return sc_decoder_push(decoder, packet);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
void
Give a name to decoder instances This will be useful in logs. PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-02-25 04:22:35 +08:00
sc_decoder_init(struct sc_decoder *decoder, const char *name) {
decoder->name = name; // statically allocated
Use frame source trait in decoder
2023-03-02 16:37:36 +08:00
sc_frame_source_init(&decoder->frame_source);
Explicitly initialize decoder sink_count The zero-initialization relied on the fact that the decoder instance is static.
2021-05-29 03:23:10 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static const struct sc_packet_sink_ops ops = {
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
.open = sc_decoder_packet_sink_open,
.close = sc_decoder_packet_sink_close,
.push = sc_decoder_packet_sink_push,
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
};
decoder->packet_sink.ops = &ops;
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
}
Reference in a new issue Copy permalink