Screen Object Finder

Tracks people on the phone screen live by drawing a bounding box around them. Works in games too 😁😉😁

The app is built as a screen-reading pipeline. It has been built, but I haven't run it on a phone yet.

• Permissions: It asks for overlay permission, notification permission, and screen-capture consent (Android MediaProjection).
• Capture: A foreground service mirrors the screen into an ImageReader through a VirtualDisplay, which supplies the frames.
• Detection: PocketBuilder's vision library runs a person detector on the full frame plus one rotating screen section per scan. It keeps only "person" results at 20% confidence or higher. Results older than about 0.9 seconds are dropped. There is no fixed pause between scans.
• Tracking: A tracker I the AI wrote in Java follows the ByteTrack approach:
• A Kalman filter predicts where each tracked box should be on the next frame.
• Detections are matched to existing tracks in two passes, confident detections first and weaker ones second, using box overlap.
• The Hungarian algorithm finds the best one-to-one matches.
• Unmatched tracks are kept for a short time, so a brief miss doesn't create a new ID.
• Drawing: A transparent overlay view draws each box in a colour tied to its ID, with a "Person #N" label and confidence.

Tested in game Free Fire 🔥🚒 See screenshots.