Commit graph

108 commits

Author SHA1 Message Date
Romain Vimont
79278961b9 Implement buffering
To minimize latency (at the cost of jitter), scrcpy always displays a
frame as soon as it available, without waiting.

However, when recording (--record), it still writes the captured
timestamps to the output file, so that the recorded file can be played
correctly without jitter.

Some real-time use cases might benefit from adding a small latency to
compensate for jitter too. For example, few tens of seconds of latency
for live-streaming are not important, but jitter is noticeable.

Therefore, implement a buffering mechanism (disabled by default) to add
a configurable latency delay.

PR #2417 <https://github.com/Genymobile/scrcpy/issues/2417>
2021-07-14 14:27:33 +02:00
Romain Vimont
408a301201 Notify new frames via callbacks
Currently, a frame is available to the consumer as soon as it is pushed
by the producer (which can detect if the previous frame is skipped).

Notify the new frames (and frame skipped) via callbacks instead.

This paves the way to add (optional) buffering, which will introduce a
delay between the time when the frame is produced and the time it is
available to be consumed.
2021-07-14 14:22:32 +02:00
Romain Vimont
336248df08 Rename video_buffer to sc_video_buffer
Add a scrcpy-specific prefix.
2021-07-14 14:22:32 +02:00
Romain Vimont
9b89b7ab72 Center the window on resize-to-fit
When removing the black borders (by double-clicking on them, or by
pressing MOD+w), the window is resized to fit the device screen, but its
top-left position was left unchanged.

Instead, move the window so that the new window area is at the center of
the old window area.

Refs #2387 <https://github.com/Genymobile/scrcpy/issues/2387>
2021-06-14 21:24:51 +02:00
Romain Vimont
7343b233e4 Render screen on window restored
It should not be necessary, since screen_render() is called just after
on SDL_WINDOWEVENT_EXPOSED, but in practice the window content might not
be correctly displayed on restored if a rotation occurred while
minimized.

Note that calling screen_render() twice in a row on
SDL_WINDOWEVENT_EXPOSED also "fixes" the issue.
2021-06-14 09:36:08 +02:00
Romain Vimont
dcee7c0f7f Factorize screen_init() error management 2021-05-16 18:48:04 +02:00
Romain Vimont
e604e8a752 Move fps_counter to screen
The FPS counter specifically count frames from the screen video buffer,
so it is specific to the screen.
2021-05-16 18:47:16 +02:00
Romain Vimont
f19c455110 Fix leak on error
Destroy video buffer if screen window creation failed.
2021-05-16 18:46:47 +02:00
Romain Vimont
0541f1bff2 Hide the window immediately on close
The screen may not be destroyed immediately on close to avoid undefined
behavior, because it may still receive events from the decoder.

But the visual window must still be closed immediately.
2021-04-25 14:38:42 +02:00
Romain Vimont
0272e6dc77 Assert screen closed on destroy
The destruction order is important, but tricky, because the screen is
open/close by the decoder, but destroyed by scrcpy.c on the main thread.

Add assertions to guarantee that the screen is not destroyed before
being closed.
2021-04-25 14:38:42 +02:00
Romain Vimont
2a94a2b119 Remove video_buffer callbacks
Now that screen is both the owner and the listener of the video buffer,
execute the code directly without callbacks.
2021-04-25 14:38:42 +02:00
Romain Vimont
e91acdb0c4 Move video_buffer to screen
The video buffer is now an internal detail of the screen component.

Since the screen is plugged to the decoder via the frame sink trait, the
decoder does not access to the video buffer anymore.
2021-04-25 14:38:42 +02:00
Romain Vimont
08b3086ffc Expose screen as frame sink
Make screen implement the frame sink trait.

This will allow the decoder to push frames without depending on the
concrete sink type.
2021-04-25 14:38:42 +02:00
Romain Vimont
2ddf760c09 Make video_buffer more generic
The video buffer took ownership of the producer frame (so that it could
swap frames quickly).

In order to support multiple sinks plugged to the decoder, the decoded
frame must not be consumed by the display video buffer.

Therefore, move the producer and consumer frames out of the video
buffer, and use FFmpeg AVFrame refcounting to share ownership while
avoiding copies.
2021-04-25 14:38:42 +02:00
Romain Vimont
c23c38f99d Move resizing workaround to screen.c 2021-04-13 22:36:59 +02:00
Romain Vimont
65c4f487b3 Set initial fullscreen from screen.c 2021-04-13 22:15:05 +02:00
Romain Vimont
c6d7f5ee96 Make screen_show_window() static
It is only used from screen.c now.
2021-04-13 22:04:38 +02:00
Romain Vimont
9826c5c4a4 Remove HiDPI compilation flag
Always enable HiDPI support, there is no reason to expose a compilation
flag.
2021-04-04 15:00:13 +02:00
Romain Vimont
cc48b24324 Simplify screen initialization
Use a single function to initialize the screen instance.
2021-03-06 22:58:03 +01:00
Romain Vimont
597c54f049 Group screen parameters into a struct
The function screen_init_rendering had too many parameters.
2021-03-06 22:58:03 +01:00
Romain Vimont
955da3b578 Remove screen static initializer
Most of the fields are initialized dynamically.
2021-03-06 22:58:03 +01:00
Romain Vimont
cb9c42bdcb Use a callback to notify frame skip
A skipped frame is detected when the producer offers a frame while the
current pending frame has not been consumed.

However, the producer (in practice the decoder) is not interested in the
fact that a frame has been skipped, only the consumer (the renderer) is.

Therefore, notify frame skip via a consumer callback. This allows to
manage the skipped and rendered frames count at the same place, and
remove fps_counter from decoder.
2021-03-06 22:58:03 +01:00
Romain Vimont
fb9f9848bd Use a callback to notify a new frame
Make the decoder independant of the SDL even mechanism, by making the
consumer register a callback on the video_buffer.
2021-03-06 22:58:03 +01:00
Romain Vimont
441d3fb119 Make video buffer more generic
Video buffer is a tool between a frame producer and a frame consumer.

For now, it is used between a decoder and a renderer, but in the future
another instance might be used to swscale decoded frames.
2021-03-06 22:58:03 +01:00
Romain Vimont
cb197ee3a2 Move fps counter out of video buffer
In order to make video buffer more generic, move out its specific
responsibility to count the fps between the decoder and the renderer.
2021-03-06 22:58:03 +01:00
Romain Vimont
76a3d9805b Inline window events handling
Now that all screen-related events are handled from screen.c, there is
no need for a separate method for window events.
2021-02-25 22:19:05 +01:00
Romain Vimont
50b4a730e3 Handle screen-related events from screen.c 2021-02-25 22:19:05 +01:00
Romain Vimont
ea2369f568 Reference video buffer from screen
This paves the way to handle EVENT_NEW_FRAME from screen.c, by allowing
to call screen_update_frame() without an explicit video_buffer instance.
2021-02-25 22:19:05 +01:00
Romain Vimont
0538e9645b Improve error handling in screen initialization
After the struct screen is initialized, the window, the renderer and the
texture are necessarily valid, so there is no need to check in
screen_destroy().
2021-02-25 22:18:51 +01:00
Romain Vimont
a566635c43 Log mipmaps error only if mipmaps are enabled 2021-02-17 09:54:03 +01:00
Romain Vimont
862948b132 Make use_opengl local
The flag is used only locally, there is no need to store it in the
screen structure.
2021-02-17 09:54:03 +01:00
Romain Vimont
c0c4ba7009 Add intermediate frame in video buffer
There were only two frames simultaneously:
 - one used by the decoder;
 - one used by the renderer.

When the decoder finished decoding a frame, it swapped it with the
rendering frame.

Adding a third frame provides several benefits:
 - the decoder do not have to wait for the renderer to release the
   mutex;
 - it simplifies the video_buffer API;
 - it makes the rendering frame valid until the next call to
   video_buffer_take_rendering_frame(), which will be useful for
   swscaling on window resize.
2021-02-17 09:54:03 +01:00
Romain Vimont
f6320c7e31 Wrap SDL thread functions into scrcpy-specific API
The goal is to expose a consistent API for system tools, and paves the
way to make the "core" independant of SDL in the future.
2021-02-17 09:54:03 +01:00
Romain Vimont
59feb2a15c Group common includes into common.h
Include config.h and compat.h in common.h, and include common.h from all
source files.
2021-01-08 19:22:10 +01:00
Romain Vimont
e8a565f9ea Fix touch events HiDPI-scaling
Touch events were HiDPI-scaled twice:
 - once because the position (provided as floats between 0 and 1) were
   converted in pixels using the drawable size (not the window size)
 - once due to screen_convert_to_frame_coords()

One possible fix could be to compute the position in pixels from the
window size instead, but this would unnecessarily round the event
position to the nearest window coordinates (instead of drawable
coordinates).

Instead, expose two separate functions to convert to frame coordinates
from either window or drawable coordinates.

Fixes #1536 <https://github.com/Genymobile/scrcpy/issues/1536>
Refs #15 <https://github.com/Genymobile/scrcpy/issues/15>
Refs e40532a376
2020-06-27 15:45:28 +02:00
Romain Vimont
0ba74fbd9a Make scrcpy.h independant of other headers
The header scrcpy.h is intended to be the "public" API. It should not
depend on other internal headers.

Therefore, declare all required structs in this header and adapt
internal code.
2020-06-19 22:30:02 +02:00
Romain Vimont
ac4c8b4a3f Increase LOD bias for mipmapping
Mipmapping caused too much blurring.

Using a LOD bias of -1 instead of -0.5 seems a better compromise to
avoid low-quality downscaling while keeping sharp edges in any case.

Refs <https://github.com/Genymobile/scrcpy/issues/1394>
Refs <https://github.com/Genymobile/scrcpy/issues/40#issuecomment-591917787>
2020-05-23 14:32:16 +02:00
Romain Vimont
fae3f9eeab Remove warning when renderer is not OpenGL
Trilinear filtering can currently only be enabled for OpenGL renderers.

Do not print a warning if the renderer is not OpenGL, as it can confuses
users, while nothing is wrong.
2020-05-23 14:23:30 +02:00
Romain Vimont
f5aeecbc62 Reset window size on initialization
On macOS with renderer "metal", HiDPI scaling may be incorrect on
initialization when several displays are connected.

Resetting the window size fixes the problem.

Refs #15 <https://github.com/Genymobile/scrcpy/issues/15>
2020-05-23 14:19:09 +02:00
Romain Vimont
e40532a376 Manually position and scale the content
Position and scale the content "manually" instead of relying on the
renderer "logical size".

This avoids possible rounding differences between the computed window
size and the content size, causing one row or column of black pixels on
the bottom or on the right.

This also avoids HiDPI scale issues, by computing the scaling manually.

This will also enable to draw items at their expected size on the screen
(unscaled).

Fixes #15 <https://github.com/Genymobile/scrcpy/issues/15>
2020-05-23 14:19:09 +02:00
Romain Vimont
d860ad48e6 Extract optimal window size detection
Extract the computation to detect whether the current size of the window
is already optimal.

This will allow to reuse it for rendering.
2020-05-23 14:19:09 +02:00
Romain Vimont
ec047b501e Disable "resize to fit" in maximized state
In maximized state (but not fullscreen), it was possible to resize to
fit the device screen (with Ctrl+x or double-clicking on black borders).

This caused problems on macOS with the "expand to fullscreen" feature,
which behaves like a fullscreen mode but is seen as maximized by SDL.
In that state, resizing to fit causes unexpected results.

To keep the behavior consistent on all platforms, just disable "resize
to fit" when the window is maximized.
2020-05-23 14:19:09 +02:00
Romain Vimont
4c2e10fd74 Workaround maximized+fullscreen on Windows
On Windows, in maximized+fullscreen state, disabling fullscreen mode
unexpectedly triggers the "restored" then "maximized" events, leaving
the window in a weird state (maximized according to the events, but not
maximized visually).

Moreover, apply_pending_resize() asserts that fullscreen is disabled.

To avoid the issue, if fullscreen is set, just ignore the "restored"
event.
2020-05-23 14:19:09 +02:00
Romain Vimont
6b1da2fcff Simplify size changes in fullscreen or maximized
If the content size changes (due to rotation for example) while the
window is maximized or fullscreen, the resize must be applied once
fullscreen and maximized are disabled.

The previous strategy consisted in storing the windowed size, computing
the target size on rotation, and applying it on window restoration. But
tracking the windowed size (while ignoring the non-windowed size) was
tricky, due to unspecified order of SDL events (e.g. size changes can be
notified before "maximized" events), race conditions when reading window
flags, different behaviors on different platforms...

To simplify the whole resize management, store the old content size (the
frame size, possibly rotated) when it changes while the window is
maximized or fullscreen, so that the new optimal size can be computed on
window restoration.
2020-05-23 14:19:09 +02:00
Romain Vimont
2608b1dc62 Factorize window resize
When the content size changes, either on frame size or client rotation
changes, the window must be resized. Factorize for both cases.
2020-05-23 14:19:09 +02:00
Romain Vimont
a14840a515 Fix typo in comments 2020-04-24 23:01:58 +02:00
Romain Vimont
8581d6850b Stabilize auto-resize
The window dimensions are integers, so resizing to fit the content may
not be exact.

When computing the optimal size, it could cause to reduce alternatively
the width and height by few pixels, making the "optimal size" unstable.

To avoid this problem, check if the optimal size is already correct
either by keeping the width or the height.
2020-04-24 22:52:02 +02:00
Romain Vimont
3c9ae99dda Move rotation coordinates to screen
Move the window-to-frame coordinates conversion from the input manager
to the screen.

This will allow to apply more screen-related transformations without
impacting the input manager.
2020-04-18 02:15:22 +02:00
Romain Vimont
44f720e4a4 Log new size on auto-resize request
On "resize to fit" and "resize to pixel-perfect", log the new size.
2020-04-18 02:15:22 +02:00
Romain Vimont
11a61b2cb3 Add option --no-mipmaps
Add an option to disable trilinear filtering even if mipmapping is
available.
2020-04-15 17:39:51 +02:00