Android/scrcpy
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity

Update DEVELOP.md

Browse source
This commit is contained in:
Romain Vimont 2019-06-11 21:16:31 +02:00
parent ffe0417228
commit 5ffdcbb7be
1 changed files with 37 additions and 38 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

75
DEVELOP.md
View file

@ -32,7 +32,7 @@ The server is a Java application (with a [`public static void main(String...
args)`][main] method), compiled against the Android framework, and executed as args)`][main] method), compiled against the Android framework, and executed as
`shell` on the Android device. `shell` on the Android device.
[main]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/Server.java#L100 [main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123
To run such a Java application, the classes must be [_dexed_][dex] (typically, To run such a Java application, the classes must be [_dexed_][dex] (typically,
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
@ -65,18 +65,18 @@ They can be called using reflection though. The communication with hidden
components is provided by [_wrappers_ classes][wrappers] and [aidl]. components is provided by [_wrappers_ classes][wrappers] and [aidl].
[hidden]: https://stackoverflow.com/a/31908373/1987178 [hidden]: https://stackoverflow.com/a/31908373/1987178
[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers [wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers
[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/aidl/android/view [aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view
### Threading ### Threading
The server uses 2 threads: The server uses 3 threads:
- the **main** thread, encoding and streaming the video to the client; - the **main** thread, encoding and streaming the video to the client;
- the **controller** thread, listening for _control events_ (typically, - the **controller** thread, listening for _control messages_ (typically,
keyboard and mouse events) from the client. keyboard and mouse events) from the client;
- the **receiver** thread (managed by the controller), sending _device events_ - the **receiver** thread (managed by the controller), sending _device messges_
to the clients (currently, it is only used to send the device clipboard to the clients (currently, it is only used to send the device clipboard
content). content).
@ -92,9 +92,9 @@ The video is encoded using the [`MediaCodec`] API. The codec takes its input
from a [surface] associated to the display, and writes the resulting H.264 from a [surface] associated to the display, and writes the resulting H.264
stream to the provided output stream (the socket connected to the client). stream to the provided output stream (the socket connected to the client).
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java [`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html [`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
[surface]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L69-L70 [surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69
On device [rotation], the codec, surface and display are reinitialized, and a On device [rotation], the codec, surface and display are reinitialized, and a
new video stream is produced. new video stream is produced.
@ -108,31 +108,30 @@ because it avoids to send unnecessary frames, but there are drawbacks:
Both problems are [solved][repeat] by the flag Both problems are [solved][repeat] by the flag
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag]. [`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90 [rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
[repeat]: [repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER [repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
### Input events injection ### Input events injection
_Control events_ are received from the client by the [`EventController`] (run in _Control messages_ are received from the client by the [`Controller`] (run in a
a separate thread). There are several types of input events: separate thread). There are several types of input events:
- keycode (cf [`KeyEvent`]), - keycode (cf [`KeyEvent`]),
- text (special characters may not be handled by keycodes directly), - text (special characters may not be handled by keycodes directly),
- mouse motion/click, - mouse motion/click,
- mouse scroll, - mouse scroll,
- other commands (e.g. to switch the screen on or to copy the clipboard). - other commands (e.g. to switch the screen on or to copy the clipboard).
All of them may need to inject input events to the system. To do so, they use Some of them need to inject input events to the system. To do so, they use the
the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our _hidden_ method [`InputManager.injectInputEvent`] (exposed by our
[`InputManager` wrapper][inject-wrapper]). [`InputManager` wrapper][inject-wrapper]).
[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/EventController.java#L66 [`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html [`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html [`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857 [`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27 [inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
@ -188,12 +187,13 @@ The client uses 4 threads:
- the **main** thread, executing the SDL event loop, - the **main** thread, executing the SDL event loop,
- the **stream** thread, receiving the video and used for decoding and - the **stream** thread, receiving the video and used for decoding and
recording, recording,
- the **controller** thread, sending _control events_ to the server. - the **controller** thread, sending _control messages_ to the server,
- the **receiver** thread (managed by the controller), receiving _device - the **receiver** thread (managed by the controller), receiving _device
events_ from the client. messages_ from the client.
In addition, another thread can be started if necessary to handle APK In addition, another thread can be started if necessary to handle APK
installation or file push requests (via drag&drop on the main window). installation or file push requests (via drag&drop on the main window) or to
print the framerate regularly in the console.
@ -217,10 +217,10 @@ to decode a new frame while the main thread renders the last one.
If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw
H.264 packet to the output video file. H.264 packet to the output video file.
[stream]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/stream.h [stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h
[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/decoder.h [decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h
[video_buffer]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/video_buffer.h [video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h
[recorder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/recorder.h [recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h
``` ```
+----------+ +----------+ +----------+ +----------+
@ -234,19 +234,19 @@ H.264 packet to the output video file.
### Controller ### Controller
The [controller] is responsible to send _control events_ to the device. It runs The [controller] is responsible to send _control messages_ to the device. It
in a separate thread, to avoid I/O on the main thread. runs in a separate thread, to avoid I/O on the main thread.
On SDL event, received on the main thread, the [input manager][inputmanager] On SDL event, received on the main thread, the [input manager][inputmanager]
creates appropriate [_control events_][controlevent]. It is responsible to creates appropriate [_control messages_][controlmsg]. It is responsible to
convert SDL events to Android events (using [convert]). It pushes the _control convert SDL events to Android events (using [convert]). It pushes the _control
events_ to a queue hold by the controller. On its own thread, the controller messages_ to a queue hold by the controller. On its own thread, the controller
takes events from the queue, that it serializes and sends to the client. takes messages from the queue, that it serializes and sends to the client.
[controller]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/controller.h [controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h
[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/control_event.h [controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/input_manager.h [inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h
[convert]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/convert.h [convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h
### UI and event loop ### UI and event loop
@ -257,10 +257,9 @@ thread.
Events are handled in the [event loop], which either updates the [screen] or Events are handled in the [event loop], which either updates the [screen] or
delegates to the [input manager][inputmanager]. delegates to the [input manager][inputmanager].
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c [scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c
[event loop]: [event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201
https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c#L187 [screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h
[screen]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/screen.h
## Hack ## Hack