Commit graph

67 commits

Author SHA1 Message Date
Romain Vimont
ae6ec7a194 Unref decoder AVFrame immediately
The frame can be unref immediately after it is pushed to the frame
sinks.

It was not really a memory leak because the frame was unref every time
by avcodec_receive_frame() (and freed on close), but a reference was
unnecessarily kept for too long.
2021-04-26 18:05:43 +02:00
Romain Vimont
8b90dc61b9 Handle EAGAIN on send_packet in decoder
EAGAIN was only handled on receive_frame.

In practice, it should not be necessary, since one packet always
contains one frame. But just in case.
2021-04-25 14:38:42 +02:00
Romain Vimont
6f5ad21f57 Make decoder push frames to sinks
Now that screen implements the packet sink trait, make decoder push
packets to the sinks without depending on the concrete sink types.
2021-04-25 14:38:42 +02:00
Romain Vimont
f7a1b67d66 Make stream push packets to sinks
Now that decoder and recorder implement the packet sink trait, make
stream push packets to the sinks without depending on the concrete sink
types.
2021-04-25 14:38:42 +02:00
Romain Vimont
cbed38799e Expose decoder as packet sink
Make decoder implement the packet sink trait.

This will allow the stream to push packets without depending on the
concrete sink type.
2021-04-25 14:38:42 +02:00
Romain Vimont
5beb7d6c02 Reorder decoder functions
This will make further commits more readable.
2021-04-25 14:38:42 +02:00
Romain Vimont
2ddf760c09 Make video_buffer more generic
The video buffer took ownership of the producer frame (so that it could
swap frames quickly).

In order to support multiple sinks plugged to the decoder, the decoded
frame must not be consumed by the display video buffer.

Therefore, move the producer and consumer frames out of the video
buffer, and use FFmpeg AVFrame refcounting to share ownership while
avoiding copies.
2021-04-25 14:38:42 +02:00
Romain Vimont
de9b79ec2d Remove compat with old FFmpeg decoding API
The new API has been introduced in 2016 in libavcodec 57.xx, it's very
old.

This will avoid to maintain two code paths for decoding.
2021-04-25 14:38:42 +02:00
Romain Vimont
55806e7d31 Remove option --render-expired-frames
This flag forced the decoder to wait for the previous frame to be
consumed by the display.

It was initially implemented as a compilation flag for testing, not
intended to be exposed at runtime. But to remove ifdefs and to allow
users to test this flag easily, it had finally been exposed by commit
ebccb9f6cc.

In practice, it turned out to be useless: it had no practical impact,
and it did not solve or mitigate any performance issues causing frame
skipping.

But that added some complexity to the codebase: it required an
additional condition variable, and made video buffer calls possibly
blocking, which in turn required code to interrupt it on exit.

To prepare support for multiple sinks plugged to the decoder (display
and v4l2 for example), the blocking call used for pacing the decoder
output becomes unacceptable, so just remove this useless "feature".
2021-04-25 14:38:42 +02:00
Romain Vimont
a09733d175 Remove useless includes from decoder.c 2021-04-10 18:48:44 +02:00
Romain Vimont
6231f683af Fix compilation error for old decoding API
Commits cb9c42bdcb and
441d3fb119 updated the code only for the
new decoding API.
2021-04-05 22:35:14 +02:00
Romain Vimont
cb9c42bdcb Use a callback to notify frame skip
A skipped frame is detected when the producer offers a frame while the
current pending frame has not been consumed.

However, the producer (in practice the decoder) is not interested in the
fact that a frame has been skipped, only the consumer (the renderer) is.

Therefore, notify frame skip via a consumer callback. This allows to
manage the skipped and rendered frames count at the same place, and
remove fps_counter from decoder.
2021-03-06 22:58:03 +01:00
Romain Vimont
fb9f9848bd Use a callback to notify a new frame
Make the decoder independant of the SDL even mechanism, by making the
consumer register a callback on the video_buffer.
2021-03-06 22:58:03 +01:00
Romain Vimont
441d3fb119 Make video buffer more generic
Video buffer is a tool between a frame producer and a frame consumer.

For now, it is used between a decoder and a renderer, but in the future
another instance might be used to swscale decoded frames.
2021-03-06 22:58:03 +01:00
Romain Vimont
cb197ee3a2 Move fps counter out of video buffer
In order to make video buffer more generic, move out its specific
responsibility to count the fps between the decoder and the renderer.
2021-03-06 22:58:03 +01:00
Romain Vimont
f6320c7e31 Wrap SDL thread functions into scrcpy-specific API
The goal is to expose a consistent API for system tools, and paves the
way to make the "core" independant of SDL in the future.
2021-02-17 09:54:03 +01:00
Romain Vimont
59feb2a15c Group common includes into common.h
Include config.h and compat.h in common.h, and include common.h from all
source files.
2021-01-08 19:22:10 +01:00
Romain Vimont
d0f5a7fd9f Remove unused includes
No mutex is used in decoder.c and stream.c.
2019-11-27 21:40:54 +01:00
Romain Vimont
510caff0cd Replace SDL_assert() by assert()
SDL_assert() open a dialog on assertion failure.

There is no reason not to use assert() directly.
2019-11-27 21:19:46 +01:00
Romain Vimont
dfd0707a29 Move utilities to util/ 2019-11-24 11:53:23 +01:00
Romain Vimont
1f8ba1ca79 Include config.h everywhere
Ref: <https://github.com/Genymobile/scrcpy/issues/829>

Suggested-by: Louis Kruger <louisk@gmail.com>
2019-09-29 22:39:53 +02:00
Romain Vimont
a7b3901c31 Add more consts
Some decoder and recorder functions must not write to AVCodec and
AVPacket.
2019-03-03 12:02:41 +01:00
Romain Vimont
8595862005 Use explicit output parameter for skipped frame
The function video_buffer_offer_decoded_frame() returned a bool to
indicate whether the previous frame had been consumed.

This was confusing, because we could expect the returned bool report
whether the action succeeded.

Make the semantic explicit by using an output parameter.

Also revert the flag (report if the frame has been skipped instead of
consumed) to avoid confusion for the first frame (the previous is
neither skipped nor consumed because there is no previous frame).
2019-03-03 00:35:20 +01:00
Romain Vimont
dfed1b250e Replace SDL types by C99 standard types
Scrcpy is a C11 project. Use the C99 standard types instead of the
SDL-specific types:

    SDL_bool -> bool
    SintXX   -> intXX_t
    UintXX   -> uintXX_t
2019-03-02 23:55:23 +01:00
Romain Vimont
aeda583a2c Update code style
Limit source code to 80 chars, and declare functions return type and
modifiers on a separate line.

This allows to avoid very long lines, and all function names are
aligned.

(We do this on VLC, and I like it.)
2019-03-02 20:28:46 +01:00
Romain Vimont
e6e011baaf Add stream layer
The decoder initially read from the socket, decoded the video and sent
the decoded frames to the screen:

              +---------+      +----------+
  socket ---> | decoder | ---> |  screen  |
              +---------+      +----------+

The design was simple, but the decoder had several responsabilities.

Then we added the recording feature, so we added a recorder, which
reused the packets received from the socket managed by the decoder:

                                    +----------+
                               ---> |  screen  |
              +---------+     /     +----------+
  socket ---> | decoder | ----
              +---------+     \     +----------+
                               ---> | recorder |
                                    +----------+

This lack of separation of concerns now have concrete implications: we
could not (properly) disable the decoder/display to only record the
video.

Therefore, split the decoder to extract the stream:

                                    +----------+      +----------+
                               ---> | decoder  | ---> |  screen  |
              +---------+     /     +----------+      +----------+
  socket ---> | stream  | ----
              +---------+     \     +----------+
                               ---> | recorder |
                                    +----------+

This will allow to record the stream without decoding the video.
2019-03-02 18:45:45 +01:00
Romain Vimont
bcd4090d51 Fix recording with old decoding/encoding API
The deprecated avcodec_decode_video2() should always the whole packet,
so there is no need to loop (cf doc/examples/demuxing_decoding.c in
FFmpeg).

This hack changed the packet size and data pointer. This broke recording
which used the same packet.
2019-03-02 17:03:15 +01:00
Romain Vimont
84270e2d18 Rename "stop" to "interrupt"
The purpose of video_buffer_stop() is to interrupt any blocking call, so
rename it to video_buffer_interrupt().
2019-03-02 17:03:15 +01:00
Romain Vimont
fff87095d9 Rename "frames" to "video_buffer"
It better describes the purpose of the structure.
2019-03-02 15:24:33 +01:00
Romain Vimont
751600a7f9 Move all compat ifdefs definitions to compat.h
This allows to give a proper name to features requirements.
2019-02-16 15:28:56 +01:00
Romain Vimont
1a5ba59504 Fix memory leak on close
The buffer associated to the AVIOContext must be freed.
2019-02-09 12:27:13 +01:00
Romain Vimont
2876463d39 Fix read_packet() return value on error or EOF
Fix warning on error or EOF:

> Invalid return value 0 for stream protocol

See <http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a606f27f4c610708fa96e35eed7b7537d3d8f712>.

Fixes <https://github.com/Genymobile/scrcpy/issues/333>.
2018-11-12 14:45:43 +01:00
Romain Vimont
22ff03f2f7 Do not queue invalid PTS
Configuration packets produced by MediaCodec have no valid PTS, and do
not produce frame. Do not queue their (invalid) PTS not to break the
matching between frames and their PTS.
2018-11-11 21:37:31 +01:00
Romain Vimont
60afb46c8d Store queue of PTS for pending frames
Several frames may be read by read_packet() before they are consumed
(returned by av_read_frame()), so we need to store the PTS of frames in
order, so that the right PTS is assigned to the right frame.
2018-11-11 21:37:31 +01:00
Romain Vimont
345f8858d3 Send frame meta only if recording is enabled
The client needs the PTS for each frame only if recording is enabled.
Otherwise, the PTS are not necessary, and the protocol is more
straighforward.
2018-11-11 21:37:31 +01:00
Romain Vimont
70579dc709 Wrap receiver state into separate struct
For readability, wrap the state of the receiver in a separate struct
receiver_state.
2018-11-11 21:37:31 +01:00
Romain Vimont
e562837c0b Avoid partial header reads
Use net_recv_all() to avoid partial reads for the "meta" header (this
would break the whole stream).
2018-11-11 21:37:31 +01:00
Romain Vimont
ebe998cf78 Move buffer reader functions to buffer_util.h 2018-11-11 21:37:31 +01:00
Romain Vimont
475912a39c Do not transmit MediaCodec flags
Since PTS handling has been fixed, the recorder do not associate a PTS
to a wrong frame anymore, so PTS of "configuration packets" (which never
produce a frame), are never read by the recorder. Therefore, there is no
need to ignore them explicitly, so we can remove the MediaCodec flags
completely.
2018-11-11 21:37:31 +01:00
Romain Vimont
27e8a9a79d Assign PTS to the right frame
The PTS was read from the socket and set as the current one even before
the frame was consumed, so it could be assigned to the previous frame
"in advance".

Store the PTS for the current frame and the last PTS read from the
packet header of the next frame in separate fields.

As a side-effect, this fixes the warning on quit:

> Application provided invalid, non monotonically increasing dts to
> muxer in stream 0: 17164020 >= 17164020
2018-11-11 21:37:31 +01:00
Romain Vimont
61db575861 Decode and push frame before recording
Handle display before recording, to reduce latency.
2018-11-11 21:37:31 +01:00
Romain Vimont
2cd99e7205 Only set valid PTS/DTS
When the PTS is valid, set both PTS and DTS to avoid FFmpeg warnings.

Since configuration packets have no PTS, do not record these packets.
2018-11-11 21:37:26 +01:00
Romain Vimont
27686e9361 Add recorder
Implement recording in a separate "class".
2018-11-11 16:30:23 +01:00
Ivan Gorinov
d706c5df39 Enable video output file, with pts set by server 2018-11-11 16:30:23 +01:00
Romain Vimont
5c739874a4 Fix memory leak on error
On decode error, unref the packet.
2018-11-09 16:10:44 +01:00
Romain Vimont
536b31829a Separate multi-words filenames by '_'
Rename foobar.ext to foo_bar.ext.

<https://github.com/Genymobile/scrcpy/pull/226#discussion_r209454865>
2018-08-15 19:30:01 +02:00
Romain Vimont
35298bb0c6 Process the last video frame
On H.264 stream EOF, the eof_reached flag is set, but av_read_frame()
still provides a frame, so check the flag only afterwards.

As a side-effect, it also fixes a memory leak (the very last packet was
not unref).
2018-03-23 14:01:58 +01:00
Romain Vimont
73c332e3e4 Unref last packet on exit 2018-03-23 13:57:32 +01:00
Romain Vimont
f9562f537a Unref the packet on error
Do not leak the packet data on error.
2018-03-08 21:36:04 +01:00
Romain Vimont
a34fbd23e9 Do not leak the packet data
Oops! The content of the packets were never freed.
2018-03-08 20:46:02 +01:00