Find image on screen

Syntax1 - find

int scan(image [window] [rect] [flags] [colorDiff] [matchIndex|array])

 

Syntax2 - wait

wait timeS [-]S image [window] [rect] [flags] [colorDiff] [matchIndex|array]

 

Parameters

image - image to find. Can be:

window - top-level or child window where to search.

rect - variable of type RECT. Default: 0.

flags:

1 Move mouse to the image. This flag not used with flag 0x200 and -S.
2 Error if image not found. This flag not used with wait.
4 Use the top-left pixel color of the image as transparent color.
16 Search only in the client area of the window. This flag is not used when searching in whole screen or bitmap. When window is an accessible object, this flag is useful with flags 0x100 and 0x1000.
32 When image is icon file, use large icon (32x32). Default - small icon (16x16).
64 If image is handle or exe resource id, use this flag to specify that it is icon.
128 QM 2.3.2. Use rect for results, but not to limit the search area.
0x100

QM 2.3.2. Get pixel colors not from screen but directly from window, with function PrintWindow.

Notes:

  • Works even if the window is in background (covered by other windows) or offscreen, but not when minimized or hidden.
  • The window may flicker, especially when waiting. The speed depends on window.
  • With this flag scan cannot find images in some windows (including Windows store apps), and in some window parts (nonclient, glass). Error if the window is of a higher integrity level process.
  • This flag is ignored: 1. If the window is DPI-scaled. 2. With flag 0x1000 if Aero is enabled.
0x200 QM 2.3.2. Search in bitmap stored in memory. Then window must be bitmap handle. If rect used, it receives coordinates of the found image in the bitmap; it is not used to limit the search area.
0x400 QM 2.3.2. When found, wait until mouse buttons released. It is more useful with wait and flag 1 (move mouse).
0x800 QM 2.4.3. Find all images specified in images. Return 1 if all found, else return 0 (or error, if flag 2).
0x1000

QM 2.4.3. Use a fast and flicker-free method to get pixel colors directly from window.

Notes:

  • Works only if Windows Aero theme is enabled. Else this flag is ignored. Also it is ignored if the window is DPI-scaled.
  • Works even if the window is in background (covered by other windows) or offscreen, but not when minimized or hidden.
  • This flag makes scan faster when Aero is enabled (Aero makes it slower). The speed depends on window and control.
  • With this flag scan cannot find images in some windows (including Windows store apps), and in some window parts (nonclient, glass).
  • On Windows XP Aero is unavailable. On Windows Vista/7 it can be enabled or disabled. On Windows 8/10 it is always enabled (now it is not called "Aero" but is used the same technology).

colorDiff (QM 2.3.2) - maximal allowed color/brightness difference of the on-screen image. Allows to find images that don't match exactly. Can be 0-255, but should be as small as possible.

matchIndex (QM 2.3.2) - 1-based index of matched image. 0 is same as 1. Use when there are several matching images in the search area and you need not the first one.

array (QM 2.3.3) - variable of type ARRAY(RECT) that will receive coordinates of all matching images.

 

timeS - max time (seconds) to wait. Error on timeout. If <=0, waits infinitely.

S - literal S.

 

Remarks

scan searches for the specified image on the screen. Returns 1 if found, 0 if not (or throws error if flag 2 is set).

 

wait waits for the specified image on the screen. Almost everything is the same as with scan. In the following text, "scan" or "the function" means "scan and wait".

 

To capture/save an image and create code, use the 'Find image' dialog. The captured image can be saved in macro resources or in a .bmp file.

 

By default the dialog saves the captured image as a resource of current macro, with name like "image:hFBDB67A2" or "image:button1" (if button1 specified in the dialog). The resource name is used in macro, like scan "image:button1" 0 0 1|2. When the macro runs, scan gets the image from resources of its macro, or from resources of a caller in the function call stack.

 

You can also select an existing resource. Supported are resources with names that begin with "image:" or end with ".bmp", ".png" or ".ico". You can use resources of other macros too, like scan "resource:<macro1>image:button1" 0 0 1|2.

 

When creating exe, QM adds macro resources to exe resources, and scan in exe uses these exe resources. If in exe you use image files instead, the files must be available where exe runs. Or you can add files to exe resources.

You can add bitmap and icon files to exe resources:

1. In "Make exe" dialog check "Auto add files...".

2. In code use resource id in file path.

 

Assume there are 3 scan with 2 images:

scan "A.bmp" 0 0 1|2
 ...
scan "B.bmp" 0 0 1|2
 ...
scan "B.bmp" 0 0 1|2

 

Add resource id:

scan ":1 A.bmp" 0 0 1|2
 ...
scan ":2 B.bmp" 0 0 1|2
 ...
scan ":2 B.bmp" 0 0 1|2

 

Then A.bmp and B.bmp will be added to exe resources. In exe will be used the resource images. If the macro runs in QM, it uses file (ignores the ":1 " part).

 

QM 2.4.3. Multiple images or colors can be specified. Then scan will search for any or all of them:

 

You can make scan faster by limiting the search area. Always specify window. Even better if it is a control or accessible object. To measure the search time, press Test in the 'Find Image' dialog.

 

To limit the search area you also can use rect (variable of type RECT). The rectangle specified in rect must be relative to the search area defined by window (screen, window/control, window/control client area (flag 16) or accessible object).

 

If you use rect for results but don't want to limit the search area, the RECT variable must be initially empty (all members 0). Or use flag 128.

 

The speed also depends on computer hardware, Windows version, whether Aero is enabled and whether the correct video driver is installed. With flags 0x100 and 0x1000 depends on window.

 

The function can find only images that are visible on the screen, unless used flag 0x100, 0x1000 or 0x200. If window used, make sure that the window is not covered by other windows. The function does not test whether the image belongs to the window. You can use act to activate the window.

 

The function can only find on-screen images that exactly match the specified image. If you use colorDiff, it can find images with slightly different colors and brightness. It cannot find images with different shapes.

 

The function may fail to find image after you change Windows theme, color scheme, skin, etc, because the background color may change. To make the function independent from such changes, try flag 4. Then it does not compare pixels that have the same color as the top-left pixel of the captured image. Usually it is background color, so if background will change in the future, the function will not fail. The function also may fail after changing: display color resolution; DPI (text size); font smoothing (if the image contains text). Also be aware of window shadows.

 

With icons you should always use colorDiff. Recommended 8. Because icons often are displayed with slightly different colors than in the icon file.

 

image also can be a bitmap or icon handle, ie when the bitmap or icon is loaded in memory. These functions can be used to get handle: LoadImage, LoadPictureFile, GetFileIcon, GetWindowIcon, CaptureImageOnScreen, CaptureImageOrColor. Later use DeleteObject for bitmap or DestroyIcon for icon.

 

QM 2.3.2: If flag 0x200 used, the function searches in bitmap whose handle is passed as argument 2 (window). The bitmap can be loaded using LoadPictureFile or other function (see above), and later must be deleted using DeleteObject. Must not be selected into a device context. Searching in bitmap is faster because the slowest part usually is getting pixels from screen or window.

 

Error if:

 

The RECT type is used to specify a rectangle.

 

type RECT left top right bottom

 

left and top - top-left corner coordinates.

right and bottom - bottom-right corner coordinates. This point actually is outside the rectangle. Rectangle width is right-left. Height is bottom-top.

 

See also: wait for image or color, pixel, id, child, acc, htm.

 

Examples

 Find image (stored in resource) in "Abc" window. If found, click, else error:
scan "image:example" "Abc" 0 1|2
lef

 Wait for image (stored in file) max 10 s:
wait 10 S "example.bmp" "Abc"

 Find image in window, and get its location:
RECT r
if(scan("image:example" "Abc" r))
	out "x=%i y=%i width=%i height=%i" r.left r.top r.right-r.left r.bottom-r.top
else out "not found"

 Find image in specified rectangle area, and get its location:
RECT r; r.left=100; r.top=100; r.right=300; r.bottom=300
if(scan("image:example" 0 r))
	out "x=%i y=%i width=%i height=%i" r.left r.top r.right-r.left r.bottom-r.top
else out "not found"