Colour & image matching

Find colours and images on screen — for things drawn into images or canvases that the element tree cannot see. Everything runs on the device without the network. Arguments go in order: what to find, how many seconds to wait, the area, and then rarely changed thresholds. Coordinates come back in points, ready for client.click.

نظرة عامة

·No need to take screenshots yourself: every lookup takes its own and releases it
·To check several things on the same frame, use keepScreen / releaseScreen or withScreen
·Write areas as 0–1 screen ratios so they survive resolution changes
·Template images are delivered as files; scripts only name them

keepScreen#

Freeze the current screen: take one screenshot and keep it, so the following colour, image and text lookups all check that frame — no new screenshots and no waiting — until releaseScreen().

typescript

keepScreen(): void;

الاستثناءات

الاستثناء	متى يحدث
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

ملاحظات

Calling it again replaces the frame with a fresh one — call it at the top of each loop round to refresh.
Remember releaseScreen(), or later lookups keep looking at the old frame. withScreen releases it for you.

مثال

javascript

keepScreen();
var dot = findColor('0xFF3B30');
var btn = findImage('btn_login.png');
releaseScreen();

انظر أيضًا releaseScreen、withScreen

releaseScreen#

Release the frame frozen by keepScreen(); lookups take fresh screenshots again. Does nothing when no frame is held.

typescript

releaseScreen(): void;

مثال

javascript

releaseScreen();

انظر أيضًا keepScreen

withScreen#

Freeze the screen, run a function, and release the frame afterwards (even if the function throws), returning the function's result. The same as keepScreen / releaseScreen, without the risk of forgetting to release.

typescript

withScreen<T>(fn: () => T): T;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`fn`	`() => T`	نعم	—	The function to run against the frozen frame.

القيمة المُعادة

النوع	الوصف
`T`	Whatever fn returns.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	fn is not a function.

مثال

javascript

var hits = withScreen(function () {
  return [findColor('0xFF3B30'), findImage('btn_login.png')];
});

انظر أيضًا keepScreen

findColor#

Find the first point that matches a colour, scanning top to bottom, left to right.

typescript

findColor(color: string, timeoutSec?: number, region?: [number, number, number, number] | null): { x: number; y: number } | null;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`color`	`string`	نعم	—	'0xRRGGBB', or '0xRRGGBB-0xTTTTTT' where the part after the dash is the per-channel tolerance.
`timeoutSec`	`number`	لا	`0`	The longest wait, in seconds. 0 means look once; above 0 it takes a fresh screenshot every 0.5 seconds until there is a match or time runs out.
`region`	`[x, y, w, h] \| null`	لا	—	Only search this area; omit it for the whole screen. When all four numbers are between 0 and 1 they are screen ratios ([0, 0.5, 1, 0.5] is the lower half and works on any resolution); otherwise they are points.

القيمة المُعادة

النوع	الوصف
`{ x, y } \| null`	The matching point in points, or null.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

var p = findColor('0xFF8E42-0x101010', 0, [0, 0.1, 1, 0.4]);
if (p) {
  client.click(p.x, p.y);
}

انظر أيضًا findMultiColor、cmpColor

findMultiColor#

Multi-point colour search: scan for an anchor colour, then verify a set of points offset from it — good for recognising a small icon.

typescript

findMultiColor(anchorColor: string, offsets: string, timeoutSec?: number, region?: [number, number, number, number] | null, similarity?: number): { x: number; y: number; score: number } | null;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`anchorColor`	`string`	نعم	—	The anchor colour, written as in findColor.
`offsets`	`string`	نعم	—	The offset points: 'dx\|dy\|0xRRGGBB-0xTTTTTT,dx\|dy\|0xRRGGBB', where dx / dy are point offsets from the anchor. The console's screenshot studio generates this for you.
`timeoutSec`	`number`	لا	`0`	The longest wait, in seconds. 0 means look once; above 0 it takes a fresh screenshot every 0.5 seconds until there is a match or time runs out.
`region`	`[x, y, w, h] \| null`	لا	—	Only search this area; omit it for the whole screen. When all four numbers are between 0 and 1 they are screen ratios ([0, 0.5, 1, 0.5] is the lower half and works on any resolution); otherwise they are points.
`similarity`	`number`	لا	`0.9`	The share of offset points that must match, 0–1; 0.9 tolerates one in ten missing. It is not a colour distance.

القيمة المُعادة

النوع	الوصف
`{ x, y, score } \| null`	The anchor position and score, or null.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

var p = findMultiColor('0xDD7A5F-0x101010',
  '29|25|0xBB454B-0x101010,58|44|0xA6363A-0x101010');
if (p) {
  client.click(p.x, p.y);
}

انظر أيضًا findColor

cmpColor#

Compare colours at fixed coordinates. There is no scanning, so it is fast — ideal for telling which screen you are on.

typescript

cmpColor(points: string, timeoutSec?: number, similarity?: number): boolean;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`points`	`string`	نعم	—	'x\|y\|0xRRGGBB-0xTTTTTT,x\|y\|0xRRGGBB', coordinates in points.
`timeoutSec`	`number`	لا	`0`	The longest wait, in seconds. 0 means look once; above 0 it takes a fresh screenshot every 0.5 seconds until there is a match or time runs out.
`similarity`	`number`	لا	`1.0`	The share of points that must match, 0–1; by default every point must.

القيمة المُعادة

النوع	الوصف
`boolean`	True when the colours match.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

if (cmpColor('205|376|0xff944b-0x101010,211|386|0xff8e42')) {
  logi('on the home tab');
}

انظر أيضًا findColor、getColor

getColor#

Read the colour at one point — handy for picking reference colours and tuning tolerances while writing a script.

typescript

getColor(x: number, y: number): { r: number; g: number; b: number; hex: string };

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`x`	`number`	نعم	—	The x coordinate in points.
`y`	`number`	نعم	—	The y coordinate in points.

القيمة المُعادة

النوع	الوصف
`{ r, g, b, hex }`	The colour channels and the 0xRRGGBB hex string.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

logi(getColor(100, 200).hex);

انظر أيضًا cmpColor

findImage#

Find an image by template matching and return the best match.

typescript

findImage(template: string, timeoutSec?: number, region?: [number, number, number, number] | null, threshold?: number): Match | null;

type Match = { x: number; y: number; width: number; height: number; centerX: number; centerY: number; score: number };

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`template`	`string`	نعم	—	The template image's file name, for example btn_login.png. Templates belong to the script and are uploaded from a screenshot in the console's Code Playground; the device downloads one automatically the first time it is used. Do not pass base64.
`timeoutSec`	`number`	لا	`0`	The longest wait, in seconds. 0 means look once; above 0 it takes a fresh screenshot every 0.5 seconds until there is a match or time runs out.
`region`	`[x, y, w, h] \| null`	لا	—	Only search this area; omit it for the whole screen. When all four numbers are between 0 and 1 they are screen ratios ([0, 0.5, 1, 0.5] is the lower half and works on any resolution); otherwise they are points.
`threshold`	`number`	لا	`0.9`	The similarity threshold, 0–1. It is a normalised cross-correlation score, insensitive to overall brightness, so 0.9 means the same thing on light and dark UIs.

القيمة المُعادة

النوع	الوصف
`{ x, y, width, height, centerX, centerY, score } \| null`	The match's position, size, centre and score (all in points), or null.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`TPL_NOT_FOUND	The template cannot be obtained: this script has no uploaded image with that file name, or the download failed — the message says which. This throws rather than returning null, because a silent null looks like an ordinary miss.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

var hit = findImage('btn_login.png', 5);
if (hit) {
  client.click(hit.centerX, hit.centerY);
}

انظر أيضًا clickImage、useTemplates、findImageAll

findImageAll#

Find every match of a template, sorted by score from high to low.

typescript

findImageAll(template: string, timeoutSec?: number, region?: [number, number, number, number] | null, threshold?: number, max?: number): Match[];

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`template`	`string`	نعم	—	The template image's file name, for example btn_login.png. Templates belong to the script and are uploaded from a screenshot in the console's Code Playground; the device downloads one automatically the first time it is used. Do not pass base64.
`timeoutSec`	`number`	لا	`0`	The longest wait, in seconds, until at least one match appears. 0 means look once.
`region`	`[x, y, w, h] \| null`	لا	—	Only search this area; omit it for the whole screen. When all four numbers are between 0 and 1 they are screen ratios ([0, 0.5, 1, 0.5] is the lower half and works on any resolution); otherwise they are points.
`threshold`	`number`	لا	`0.9`	The similarity threshold, 0–1. It is a normalised cross-correlation score, insensitive to overall brightness, so 0.9 means the same thing on light and dark UIs.
`max`	`number`	لا	`10`	The most matches to return; at least 1.

القيمة المُعادة

النوع	الوصف
`{ x, y, width, height, centerX, centerY, score }[]`	An array of matches, empty when there is none.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

var stars = findImageAll('star.png', 0, null, 0.85, 5);
logi('stars: ' + stars.length);

انظر أيضًا findImage

clickImage#

Wait for a template and tap it, at a slightly random spot inside the match so it is never the same point twice.

typescript

clickImage(template: string, timeoutSec?: number, region?: [number, number, number, number] | null, threshold?: number): boolean;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`template`	`string`	نعم	—	The template image's file name, for example btn_login.png. Templates belong to the script and are uploaded from a screenshot in the console's Code Playground; the device downloads one automatically the first time it is used. Do not pass base64.
`timeoutSec`	`number`	لا	`10`	The longest wait, in seconds. It takes a fresh screenshot every 0.5 seconds until there is a match or time runs out.
`region`	`[x, y, w, h] \| null`	لا	—	Only search this area; omit it for the whole screen. When all four numbers are between 0 and 1 they are screen ratios ([0, 0.5, 1, 0.5] is the lower half and works on any resolution); otherwise they are points.
`threshold`	`number`	لا	`0.9`	The similarity threshold, 0–1. It is a normalised cross-correlation score, insensitive to overall brightness, so 0.9 means the same thing on light and dark UIs.

القيمة المُعادة

النوع	الوصف
`boolean`	True when tapped, false when the template never appeared.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	An argument is wrong: the wait is not a number or exceeds 600, the region is not [x, y, w, h], a threshold is outside 0–1, an old-style {} was passed, and so on.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

ملاحظات

By default the task log records whether the image was tapped or not found; turn it off with setClickLog(false).

مثال

javascript

if (!clickImage('btn_login.png')) {
  return false;
}

انظر أيضًا findImage、clickText

useTemplates#

Get template images ready up front. Optional: findImage downloads a template automatically the first time it is used. Calling it at the top of the script reports missing images as soon as the run starts and avoids a pause on the first search.

typescript

useTemplates(names: string[] | string): boolean;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`names`	`string[] \| string`	نعم	—	An array of file names (a single name is fine too). Names refer to the templates uploaded to this script; every script has its own, so images with the same name in different scripts never get mixed up.

القيمة المُعادة

النوع	الوصف
`boolean`	True when every template is in place; false if any is missing, with which one and why (not uploaded, or the download failed) in the error log. An image the device has cached before is not downloaded again.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`INVALID_ARGUMENT	Something other than file names was passed (such as an old-style [{ name, url }]). Use downloadTemplate to download from another address.
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

if (!useTemplates(['btn_login.png', 'btn_close.png'])) {
  return false;
}

انظر أيضًا downloadTemplate、hasTemplate

hasTemplate#

Whether this template is ready in the current run (already used, or prepared by useTemplates).

typescript

hasTemplate(name: string): boolean;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`name`	`string`	نعم	—	The file name.

القيمة المُعادة

النوع	الوصف
`boolean`	True when ready.

مثال

javascript

if (!hasTemplate('btn_login.png')) {
  loge('template missing');
}

انظر أيضًا useTemplates

downloadTemplate#

Download an image from any address as a template under a name of your choice; findImage can use that name for the rest of the run. Templates uploaded to this script don't need it — just use the file name.

typescript

downloadTemplate(url: string, name: string): boolean;

المعاملات

الاسم	النوع	مطلوب	القيمة الافتراضية	الوصف
`url`	`string`	نعم	—	The address; one starting with / is resolved against the server.
`name`	`string`	نعم	—	The name to give the image, used with findImage. Valid for the current run only.

القيمة المُعادة

النوع	الوصف
`boolean`	True once in place; false on failure, with an error log line.

الاستثناءات

الاستثناء	متى يحدث
`WDAError`FEATURE_UNAVAILABLE	The app on the device is too old to support colour and image matching.

مثال

javascript

downloadTemplate('https://example.com/btn_login.png', 'btn_login.png');

انظر أيضًا useTemplates