Is there any way (in any language) in which an image can be detected on the screen and when it is detected, perform an action? having an image of the object or thing that you want to detect (previously saving it), so that it can compare it.
an easy example to express myself: a game
on the screen it has different menus and objects, the idea is to be able to detect that "x" thing and click on the "y" thing, obviously having the option of being able to create conditions (for example, if the object "J" is on the screen and the object "P" does not perform the action, otherwise yes), but all based on what is on the screen.
I completely do not know if it is possible, I searched but did not find anything similar.
Thanks in advance!
The easiest way I can think of is to use the
pyautogui
Python library (you will surely find equivalent libraries in other languages).This solution may or may not work for you depending on the characteristics of the program you develop: if you need a lot of speed and how easily recognizable the objects are (if the background on which it is located changes, etc.). Besides that the objects always have to be in the same position and size.
The function
locateOnScreen
can take the parameterconfidence
that allows you to set a threshold of accuracy when recognizing the object (you need to install OpenCV). If the object is not found, it will returnNone
and if it is located it will return an objectBox
with the coordinates and dimensions.You can find more features here .
On the contrary, if it does not fit the needs of your project, you may have to resort to computer vision libraries such as OpenCV or SimpleCV or artificial intelligence algorithms.