From 5ffdcbb7beea8721350280170365aa8417c38ab8 Mon Sep 17 00:00:00 2001 From: Romain Vimont Date: Tue, 11 Jun 2019 21:16:31 +0200 Subject: [PATCH] Update DEVELOP.md --- DEVELOP.md | 75 +++++++++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 38 deletions(-) diff --git a/DEVELOP.md b/DEVELOP.md index ba9e130b..dea8137d 100644 --- a/DEVELOP.md +++ b/DEVELOP.md @@ -32,7 +32,7 @@ The server is a Java application (with a [`public static void main(String... args)`][main] method), compiled against the Android framework, and executed as `shell` on the Android device. -[main]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/Server.java#L100 +[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123 To run such a Java application, the classes must be [_dexed_][dex] (typically, to `classes.dex`). If `my.package.MainClass` is the main class, compiled to @@ -65,18 +65,18 @@ They can be called using reflection though. The communication with hidden components is provided by [_wrappers_ classes][wrappers] and [aidl]. [hidden]: https://stackoverflow.com/a/31908373/1987178 -[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers -[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/aidl/android/view +[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers +[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view ### Threading -The server uses 2 threads: +The server uses 3 threads: - the **main** thread, encoding and streaming the video to the client; - - the **controller** thread, listening for _control events_ (typically, - keyboard and mouse events) from the client. - - the **receiver** thread (managed by the controller), sending _device events_ + - the **controller** thread, listening for _control messages_ (typically, + keyboard and mouse events) from the client; + - the **receiver** thread (managed by the controller), sending _device messges_ to the clients (currently, it is only used to send the device clipboard content). @@ -92,9 +92,9 @@ The video is encoded using the [`MediaCodec`] API. The codec takes its input from a [surface] associated to the display, and writes the resulting H.264 stream to the provided output stream (the socket connected to the client). -[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java +[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java [`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html -[surface]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L69-L70 +[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69 On device [rotation], the codec, surface and display are reinitialized, and a new video stream is produced. @@ -108,31 +108,30 @@ because it avoids to send unnecessary frames, but there are drawbacks: Both problems are [solved][repeat] by the flag [`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag]. -[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90 -[repeat]: -https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148 +[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90 +[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148 [repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER ### Input events injection -_Control events_ are received from the client by the [`EventController`] (run in -a separate thread). There are several types of input events: +_Control messages_ are received from the client by the [`Controller`] (run in a +separate thread). There are several types of input events: - keycode (cf [`KeyEvent`]), - text (special characters may not be handled by keycodes directly), - mouse motion/click, - mouse scroll, - other commands (e.g. to switch the screen on or to copy the clipboard). -All of them may need to inject input events to the system. To do so, they use -the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our +Some of them need to inject input events to the system. To do so, they use the +_hidden_ method [`InputManager.injectInputEvent`] (exposed by our [`InputManager` wrapper][inject-wrapper]). -[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/EventController.java#L66 +[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81 [`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html [`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html [`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857 -[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27 +[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27 @@ -188,12 +187,13 @@ The client uses 4 threads: - the **main** thread, executing the SDL event loop, - the **stream** thread, receiving the video and used for decoding and recording, - - the **controller** thread, sending _control events_ to the server. + - the **controller** thread, sending _control messages_ to the server, - the **receiver** thread (managed by the controller), receiving _device - events_ from the client. + messages_ from the client. In addition, another thread can be started if necessary to handle APK -installation or file push requests (via drag&drop on the main window). +installation or file push requests (via drag&drop on the main window) or to +print the framerate regularly in the console. @@ -217,10 +217,10 @@ to decode a new frame while the main thread renders the last one. If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw H.264 packet to the output video file. -[stream]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/stream.h -[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/decoder.h -[video_buffer]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/video_buffer.h -[recorder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/recorder.h +[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h +[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h +[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h +[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h ``` +----------+ +----------+ @@ -234,19 +234,19 @@ H.264 packet to the output video file. ### Controller -The [controller] is responsible to send _control events_ to the device. It runs -in a separate thread, to avoid I/O on the main thread. +The [controller] is responsible to send _control messages_ to the device. It +runs in a separate thread, to avoid I/O on the main thread. On SDL event, received on the main thread, the [input manager][inputmanager] -creates appropriate [_control events_][controlevent]. It is responsible to +creates appropriate [_control messages_][controlmsg]. It is responsible to convert SDL events to Android events (using [convert]). It pushes the _control -events_ to a queue hold by the controller. On its own thread, the controller -takes events from the queue, that it serializes and sends to the client. +messages_ to a queue hold by the controller. On its own thread, the controller +takes messages from the queue, that it serializes and sends to the client. -[controller]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/controller.h -[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/control_event.h -[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/input_manager.h -[convert]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/convert.h +[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h +[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h +[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h +[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h ### UI and event loop @@ -257,10 +257,9 @@ thread. Events are handled in the [event loop], which either updates the [screen] or delegates to the [input manager][inputmanager]. -[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c -[event loop]: -https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c#L187 -[screen]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/screen.h +[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c +[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201 +[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h ## Hack